Events, logging, and compliance

Compliance is often cited as the reason to keep, report on, monitor or otherwise use logs and event data. But there is a lot of misunderstanding about how the compliance mandates relate to logs and event data, frequently misinformation from legacy vendors. This article clears that up with a comprehensive explanation.

---

TL;DR Logging vendors cite compliance as reasons to buy their products but have and promote a very simplistic and often misinformed view as to how and why events and logs relate to compliance. This sets the story straight and puts you on a path to a holistic approach to logging for compliance.

---

Compliance has been a driver of the adoption of log management and security event management (SEM) / security information management (SIM) — together now known as SIEM — for nearly 25 years. In this time the regulations, mandates and standards with which organizations of many kinds in many verticals and geographies must comply with or choose to comply with have multiplied and evolved significantly in terms of their drivers, scope, and the potential consequences of non-compliance.

Throughout this history, there has been a lot of confusion and overly simplistic thinking and messaging about what actually needs to be done and why, with conflation of very different uses of logs and events in compliance practices, very different objectives for different mandates and very different drivers and consequences for non-compliance.

For example, people say “SOX requires me to keep my logs.” SOX, or Sarbanes-Oxley, as a statute, makes no mention of logs. But it does bring IT risk into the purview of overall audit and risk management for public companies, which then gets translated in practice through SEC as the regulator to translate law into specific practices, which then turns to the Public Company Accounting Oversight Board (PCAOB, affectionately called “Peek-a-boo”, a nonprofit established by Congress) to create very specific controls for IT and security practices that get verified by auditors. In the end, these may specify requirements for log retention and review.

This document is intended to provide a clear understanding of the different ways that IT operations, security and privacy compliance involve log and other event data to guide compliance messaging, product priorities and design for Axiom. Compliance is one business driver of many for organizational adoption.

Discrete authorities

First off, not all compliance practices that involve logs and events are driven by laws. But those laws include:

• Sarbanes-Oxley, which applies to publicly traded companies.
• HIPAA (Health Insurance Portability and Accountability Act — it’s not HIPPA, nor does its P stand for privacy), which applies to healthcare providers and payers.
• FISMA (the Federal Information Security Management Act), which applies to US government agencies and certain contractors.
• FedRAMP (Federal Risk and Authorization Management Program), which applies to cloud providers serving the US federal government.
• GDPR (EU General Data Protection Regulation), which applies to organizations that are based in the EU and control process personal data, including digital records of digital or physical activities, or control or process personal data of EU residents.
• CCPA (California Consumer Privacy Act) and CPRA (California Privacy Rights Act), two successive laws that mandate GDPR-like regulations on for-profit companies that do business in California.

Other compliance mandates may be imposed by private organizations that gain authority not through law, but through their ability to discontinue services critical to an organization’s business, or through their ability to directly withdraw significant business for noncompliance. The most notable of these is the Payment Card Industry Data Security Standard (PCI-DSS), a very prescriptive standard for data security practices focused on consumer credit card data protection enforced by the Visa, Mastercard associations, and their member banks. They gain authority by exerting control over whether merchants can accept their cards.

Regulatory objectives

Many people loosely associate compliance with security and don’t think too much more deeply about the diverse objectives behind the laws, regulations, mandates and standards with which organizations may need to comply. It’s important to understand the spirit of compliance to have a good compliance program in order to show good faith and due care in the course of audits, enforcement and any litigation that may arise.

There are several parallel objectives:

• IT system integrity. This is the objective of Sarbanes-Oxley, which is concerned with the accuracy of financial reporting that may be impacted by internal controls dependent on IT systems that are concerned both with simple IT operations and security risk management. This IT risk management and auditing has taken its place in the last two decades alongside traditional financial risk management, such as ensuring that receivables are sound in traditional financial audits.
• Protection of sensitive private data. This is what is behind PCI-DSS (consumer credit card info), HIPAA (electronic patient health information, or EPHI, adopted to encourage the move to digital vs paper patient records by ensuring their privacy and integrity), and GLBA (sensitive financial information on consumers held by banking institutions.)
• Extension of the protection of “real world” private information. This applies to financial and health records, and to account information and the trail of a user of digital services’ activity. This is what GDPR and its many like digital privacy regulations cover, with firm requirements that deleted user records be permanently erased and that multiple records can’t be collated to deduce a user’s identity.
• Extension to anyone who touches the data. Enterprise and government customers often want to also ensure that contractors, third-party data processors, and various service providers are known to follow the same operational, security and privacy practices as they do internally. This is behind FedRAMP for cloud providers that service government customers, and SOC 1 & 2 as auditing standards for service providers, to show compliance with documented elective controls to their customers.

Organizational drivers

Enterprises and governments also adopt security frameworks, whose execution falls under the compliance header, motivated by their own assessment of business and public interest risks from threats. These include:

• Sensitive data and IP theft by competitors and hostile states.
• Critical infrastructure disruption by hostile states and terrorist organizations.
• Ransomware and theft of marketable consumer private data by financially motivated hacker gangs.
• Theft of computing resources and crypto assets.

There are different drivers for organizations to choose to comply with different mandates and the seriousness with which they take compliance with each. When positioning compliance as a reason to undertake an initiative or purchase, it’s important to understand the real business risks associated with a given mandate for a given organization:

• Access to big market segments. If you are not FedRAMP compliant, you will not be able to offer cloud services to any government agency, with very few exceptions. If you do not get a SOC 2 audit and have that audit verify a generally accepted set of compliance controls (usually by adopting a widely accepted framework like ISO 27001 or NIST 800-53 voluntarily), you will not be able to sell a B2B SaaS to most large enterprises — especially in verticals or serving verticals subject to their own compliance standards.

• Access to business-critical services. PCI-DSS’ potential to deny a business the ability to accept credit cards is the best, if extreme example. It’s a complete killer for any direct-to-consumer business of any kind.

• Reputational impact. The reporting of data breaches to impacted consumers and various agencies — such as Ticketmaster notifying past customers via US Mail — is mandated by some US states and the EU under GDPR. Such reports can have a significant negative reputation impact and thus impact business results.

• Fines. Regulators rarely impose fines for simple noncompliance. But some do impose specific fines if there is a breach or other incident that the regulation is intended to prevent and a controls failure is found to be the cause.

  These fines are often not large enough to be a major motivator unless backed up with a risk of criminal prosecution, or of civil litigation with potential major judgments. But Sarbanes-Oxley fines can reach $2.5 million for company noncompliance and $5 million for C-level execs convicted of fraud to cover up controls gaps. And the EU specifies fines of up to €20 million or 4% of global revenues, which motivates large organizations like Microsoft to take GDPR extremely seriously.

• Follow-on judgments. The fines imposed by the applicable regulatory agency, Health and Human Services, are fairly modest, with caps per organization per violation category. However, state attorneys general have pursued cases in the tens of millions of dollars for HIPAA violations, claiming negligence of duty of care. The top civil settlement was $115M for Anthem in 2018, based on expected controls that were found not to be in place in an investigation by HHS’s Office for Civil Rights, which had imposed its own $16M fine. Arguably, Anthem would have avoided steep financial penalties if they had adopted any accepted framework and documented following it.

• Criminal prosecution. Sarbanes-Oxley also provides a penalty of up to 20 years in prison for C-level execs convicted of fraudulent coverups or lack of disclosure of controls gaps. This was most recently and notably employed in 2023 to prosecute Solar Winds’ CSO for covering up security controls gaps and downplaying a major breach. Separately, the CSO of Uber was sentenced to 3 years probation after prosecutors requested a 15-year sentence for conspiring to conceal, rather than disclose, a breach of Uber driver financial records as required under California laws.

• Civil litigation. State attorneys general in the US have successfully won judgments on behalf of citizens in cases of significant data breaches tied to HIPAA controls violations that dwarf the fines directly imposed by the Health ans Human Services violations. Cottage Health was ordered to pay $2 million after patients’ medical information was exposed online.

• Direct security risks. As enumerated above, these range from data theft by competitors to ransomware by hackers.

Framework specifics

As noted earlier, few laws directly require specific controls or practices in a self-contained way. And many mandates come not from legal authorities but from private organizations — either critical business service providers like the credit card networks, or enterprise and government customers that make compliance with various frameworks or specific controls a condition for doing business.

• PCI-DSS is pretty self-contained as its own framework, but the controls it specifies are fairly common to more extensive and general standards for information security such as NIST 800-53 and ISO 27001 - with the exception of some controls very specific to protecting against credit card fraud, such as storage of security codes with credit card numbers.

• HIPAA as a law authorized the department of Health and Human Services to issue rules for the security and privacy of EPHI and PHI, which it did as the HIPAA Security Rule and the separate HIPAA Privacy Rule. These rules are frameworks unto themselves with specific security and other controls to follow and while vary similar in many places to general security standards like NIST 800-53 and ISO 27001, are completely independent and vary in certain specifics.

• ISO 27001 and NIST 800-53 are the most commonly used general IT security frameworks, specifying a wide range of controls encompassing all areas of standard healthy security practices from firewalls and identity to logging and audit. Often, these standards are chosen voluntarily by organizations in cooperation with their auditors to demonstrate due care in complying with more generally worded regulations or as the standard against which SOC 2 audits are carried out.

  ISO 27001 is the standard to comply with security requirements under GDPR. NIST 800-53 is followed by US government agencies and their contractors to comply with FISMA and FedRAMP.

• SOC. The System and Organization Control examinations, SOC 1 and SOC 2, were developed by the American Institute of Certified Public Accountants. They are neither law nor framework, but rather professional standards to validate and attest that a company follows whatever it claims are its controls and organization. You could have a SOC 2 audit that validates that you print and burn your log events once a day. More likely you would use your SOC 2 audit to validate that you have followed accepted best practices and do what ISO 27001 says.

• CMMC. The US Department of Defense has declared its Cybersecurity Maturity Model Certification a top priority. CMMC, which has a tiered model of requirements mapped to increasing levels of data sensitivity, plus specific guidelines for assessment and implementation, is designed to enforce the protection of sensitive unclassified information that the DoD shares with its contractors and subcontractors.

Compliance-driven practices

At last we get to logs and events themselves. The various frameworks adopted to comply with the various mandates specific a wide array of operations, security and privacy controls. Some are manual procedures, some are specific automated controls. They cover all aspects of IT from software development, testing and delivery to systems architecture to access controls to networking and everything else under a CIO’s and CISO’s purview.

Specific controls that involve using or handling logs or other IT system events are a small minority of these controls and fall into a few groupings. But logs as records of system and human behavior separately provide evidence for auditors and continuous internal controls monitoring of many, if not most, other controls. That’s why log-based reporting, while not mandated by any framework, forms a very important practice for a healthy overall compliance program.

To get into specifics, these are the major log-involved practices driven by compliance:

Log / audit trail retention

Some frameworks explicitly require the retention of certain audit logs for specific periods. Sarbanes-Oxley requires seven years’ retention for audit records, which is interpreted to include audit logs for access to financial systems. HIPAA requires six years for audit logs of access to personal health information. NIST and ISO vaguely require retention as long as necessary for an organization’s specific objectives. Generally organizations and auditors allow a lot of selectivity of what events are subject to extended retention. Many organizations opt for broader retention to ensure that they're covered in case of unforeseen needs.

Log / audit trail / security event review

The various frameworks describe different practices at different levels of detail for routine review of logs, audit trails, security events, etc. These can variously be satisfied by a formalized process for manual review, by reviewing and acknowledging events prioritized automatically, or by an automated review. Reports on the logs of the systems in which reviews are conducted is in turn proof that this review control is being done.

Incident investigation

The various security frameworks generally specify that incident reports and alerts of possible intrusions or breaches be investigated systematically. Of course, this usually involves logs. The frameworks may specify that the investigations seek to identify how the attacker both attempted and was able to infiltrate the system, and the nature and extent of the actual impact, culminating in measures to prevent further similar incidents and contain or remediate any damage. This all generally involves analysis of log and other event data from a wide range of systems — from security technologies like intrusion detection systems and endpoint detection and response, to networking, to applications.

SIEM

Some security frameworks explicitly specify particular technologies, such as SIEM, usually without strict definition of what constitutes a SIEM. As a baseline, implementation of any system that can raise alerts on likely intrusions based on rules applied to raw log and other events is considered a SIEM by auditors validating against such proscriptive frameworks. Organizations with established security compliance programs may have written specific vendors and their capabilities into their organization-specific controls against which their auditors perform validation.

Logs as proof of practice

As noted in the intro to this section, log event-based reporting is a sort of Swiss Army knife that can produce reports to show other controls are working. Do you have a control that says all UDP syslog is blocked? Run a report on UDP 514 firewall traffic.

Logging vendors over the last 20 years have produced elaborate report packs that map such general security controls to specific frameworks and references. All these packs are fairly generic except for their mapping to particular control framework names and numbered controls, or paragraphs or similar.

As part of a continuous controls monitoring practice, internal auditors, compliance teams or security or IT teams may make it a practice to review these reports regularly and “check them off.” This in turn becomes a control, recorded in a log event that can be reported.

Users’ right to be forgotten

GDPR in 2016 introduced a major new wrinkle to log events and compliance: The right to be forgotten. RTBF rules, which continue to proliferate geographically, mandate that anyone whose personal data is protected by the law can request for it to be completely and irrevocably erased quickly in specific circumstances. Personal data under GDPR includes any digital activity that can be associated to an individual. So the new law decreed that organizations must be able to selectively delete log events as of May 2018.

Six years later, many companies and vendors are still struggling toward compliance. The problem is that records in databases optimized for efficiency often aren’t literally deleted, but marked to be ignored. They’re still there. Copies may be scattered in an uncertain number of multiple locations. Older records may be in cold storage, where finding one user’s every entry isn’t a simple search.

Another previous approach to anonymity, depersonalization — the replacement of ID tokens with other data while leaving the rest of the event intact — also fails to meet requirements that the identities of those forgotten can’t be deduced by collating multiple records. The ease of doing so was proven 20 years ago when AOL published “anonymized” search records in which they’d replaced usernames with numeric IDs. User 44177449 was quickly identified by Internet sleuths as a 62-year-old widow in Georgia, outing her entire AOL search history in the released data. (I must note this risk likewise makes ethical behavioral analytics of product usage challenging.)

Worse, new regulations can conflict with existing compliance requirements that the same records be kept intact for years. It’s such a minefield that Axiom enlisted a lawyer with hard experience to explain how companies should set up a log governance strategy to avoid missteps to one side or the other.

The inevitable AI angle

It would be naive to think GDPR was the last and highest bar to meet for events, logging and compliance. With AI applications that feed on events as training data both exploding and evolving, it’s only a matter of time before new regulations and industry practices become business requirements, with logs as both validation and liability in new situations we haven’t yet thought of. The US, EU, China, California and other regions are in the process of developing laws to govern data lineage tracking (to maintain quality and trustworthiness of data used for training) and explainable AI (to maintain transparency and fairness while reducing bias and risk.)

The compliance challenge to all of us — engineers, vendors and executives — is to keep looking for ways to stay ahead of the game.

---

Interested to learn more about Axiom?

100% of your data for every possible need: o11y, security, analytics, and new insights.

Sign up for free or contact us at sales@axiom.co to talk with one of the team about our enterprise plans.