The AWS use for the term artifact for its new portal is the most relevant for this guide. In this guide, we walk through the importance of compliance artifacts for an effective compliance program. Compliance artifacts are often synonymous with compliance evidence. Using this definition, this is a simple topic. But compliance artifacts in the context of the cloud are anything but simple. In today’s world of cloud services, managed services, and APIs, the breadth of artifacts is massive. This article also touches on data artifacts that are being shed, often unknowingly, by cloud services and can contain protected data that needs to be handled as all other protected information is handled.
Introduction to Evidence
When it comes to compliance, a compliance maturity model is a good place to start to appreciate the value of evidence. While privacy defines the high-level policies for the management of data and technology, and security is the implementation of those privacy-driven policies, compliance is the effective measurement of the entire compliance program, from policies through secure implementation. Measurement is all about documentation, ideally on an ongoing basis, in support of a functioning compliance program and the following of compliance policies.
Hopefully, compliance is not seen as a checkbox within your organization. When compliance is seen as a roadblock or something that is bolted on after the fact, evidence is generated and stored for an auditor. It is not seen as having value and it is not used internally to monitor and improve compliance posture. Evidence should be used to constantly internally monitor your compliance posture and ensure proactive identification of gaps, threats, and risks that can then be mitigated. A mature compliance program does not simply implement secure technology configurations and internal processes, it actively manages those configurations and implementations to proactively identify and resolve gaps, find areas for improvement, and, something we’ve seen at Datica, identify where there might be human resource gaps.
Compared to the post-cloud world of today, technology choices and workflows were previously limited. Similarly, in a pre-cloud world, compliance evidence was more confined. Today, the cloud opens up a new world of compliance evidence in the form of both manually generated artifacts (most cloud providers can generate certain data and reports on demand) and automatically generated artifacts (APIs, logs, event streams, etc). The compliance-relevant artifacts generated by cloud services are not well understood or appreciated, either by the users of the cloud or by auditors and assessors.
Examples of Compliance Artifacts
With all the attention on compliance and the need for compliance artifacts, or evidence, to be successful, it’s helpful to see common examples. The list below is an example list from Datica. It is certainly not exhaustive and there are different tools to accomplish the same things.
- Jira Tickets. We are heavy Atlassian users at Datica. In addition to general company documentation, we use Confluence for documenting and change control for our procedures. When it comes to artifacts, we leverage Jira tickets and projects to ensure we document how we follow procedures and provide a historical record, with all interactions of users, for all compliance related tasks. We use templating in Jira to minimize variability in compliance procedure workflows. Jira provides good integrity to ensure logging of activities on tickets.
- Event Reviews. We have certain regular review activities we do per our policies. These are scheduled at some cadence from weekly to monthly up to annually. The most common examples are things like reviewing logs, vulnerability scan, virus scans, system inventory, etc.
- Deployment Tickets. We do security reviews as a part of our deployment ticket process. This aligns our operations with our configuration management policies and provides a separation of duties between security (reviewers) and operations.
- Workstation Configurations. We standardize on MacOS machines at Datica and use Jamf to manage those machines remotely and ensure hardened configurations are in use. Jamf reports are reviewed and logged in Jira but the actual report is also an artifact.
- Support Tickets. Some of the events that are triggered by our scans are relevant to our customers and, in those cases, we sometimes use Zendesk to communicate with them. This means we’ve had to use Zendesk tickets at times for compliance artifacts.
- Cloud Configurations. One of the new risk vectors with the cloud is configuring managed services from cloud service providers to comply with our compliance policies. This is very much an emerging area, one we go deep into in our new book on cloud compliance. The challenge is not simply setting parameters correctly for cloud services (things like encryption and open ports and backups) but also ensuring those configurations don’t change. The configuration themselves would ideally be snapshotted to create a historical record of evidence. Some of this can be done today with APIs but much of it is still manual and bespoke for each cloud service and each cloud provider.
- Events. We document our event reviews in Jira, as stated above, but the content of the events themselves are also artifacts. For logging, we leverage the ELK stack for immediate, shorter-term log storage and access. By policy, we move log data to AWS S3 and retain that data for at least a year.
- System Inventory. Doing a system inventory is a lot more difficult on the cloud, especially if you have a fleet of hundreds or thousands of instances and leverage managed services (like S3 above). In the good old days of data centers and purchased hardware, tracking sheets with serial numbers were the norm. Today, with virtualization, managed cloud services, and the on-demand nature of the cloud, system inventories are harder and harder grok, especially if you need accurate historical records. At Datica, we focus on retaining our monthly inventory, based on a mix of sources including cloud billing information.
- Network Diagrams. This can also be more challenging on the cloud given the nature of virtualization and abstraction. We manage a lot of different cloud accounts and environments at Datica and typically try to enforce a standard network architecture across all of them. This network architecture is what we maintain and it is validated by our system inventory.
- People Operations. Compliance is not just about technology. Some of the content that would historically be a part of an employee file also contains relevant compliance artifacts.
- Employment Agreements. Not all employment agreements are relevant for our compliance program and our HITRUST Certification, but some, like our IP agreements, are relevant and we have to provide them during our audits.
- Background Checks on Personnel. We conduct background checks on all new employees. We persist these reports as artifacts for future reference.
- Security Training. HIPAA, and most other compliance regimes, require regular security training. In reality, given the nature and value of digital data today and the role all companies play as stewards of that data, security training should be done at every organization, ideally tailored to the specific organization. At Datica, we require annual training and we document using G Suite (from Google). We also provide open source HIPAA training if you’re looking for starter content for your own organization.
- Patch Testing. Despite the adoption of the cloud and abstracted services from cloud providers (some levels of abstraction include the OS and software packages so patching falls on cloud providers), unpatched software is and remains one of the most relevant threat vectors. Patchers should be tested before going out to production systems and this patch testing should be documented, along with the ultimate patch information metadata when the patch is actually installed into production. Jira is a good tool for this as well.
- SLAs. We have SLAs with the various cloud service providers (CSPs). As a part of our HITRUST Certification, we are required to assess the performance of CSPs against their stated and committed service levels. We do this on an annual basis and we document it using Jira tickets.
- Risk Assessments. Risk assessment, and risk generally, should be the driving force behind many compliance and security decisions. Full risk assessments should be done at least annually while mini-risk assessments, those that evaluate significant, and sometimes even small, changes to your technology and compliance program should be done on an ad hoc basis as needed. At Datica, we leverage a risk assessment template in a Google Sheet that is based on NIST standards for risk assessments.
How to Use Artifacts
Artifacts should be used to 1) internally monitor and measure the performance of your compliance program and 2) externally validate your compliance program for auditors and other third parties. In both uses, organization and regular pruning of your compliance artifacts are essential (pruning doesn’t mean deleting).
The major challenge we’ve experienced at Datica is compliance artifact overload and consistent organization, especially with personnel changes (growth and departures). Your compliance program, and the associated evidence, should not be tied to any one individual and should outlast your team. This is easier said than done.
As you can see above, there are a lot of different forms of artifacts from the documentation of reviews (we use Jira for this) to logs to scans to SLAs to personnel files and more. We’ve tried multiple times to consolidate our tooling but, in the end, we still have a mix of Jira, Zendesk, AWS S3, ELK (on CSP), and G Suite. This highlights the volume and variety of compliance artifacts for a modern cloud company and the associated challenge of organizing those artifacts. We still use regular manual grooming to ensure things are in the right places.
Since we’ve been unable to consolidate all of our artifacts into one tool, the key is to document, ideally within the procedures themselves, where each type of compliance artifact is stored and to ideally ensure there is a separation between those that create/store the artifacts and those that access/organize/prune the artifacts.
A Caveat of Securing Artifacts (and risks)
Backups, APIs, logs, scans, tickets and more can and do create a lot of data exhaust. This data exhaust can be in the content itself or in the metadata associated with the artifacts. This data exhaust can contain PHI (logs are the easiest example to imagine this being the case) and, as such, needs to be secured and treated like any of PHI to which you are responsible. That means BAAs should be in place with any relevant tools/tech that are storing compliance artifacts and other protections, per your organizations’ policies, should be implemented.
Starting is the Hardest Part
Effectively capturing and managing compliance artifacts is not fun. And it’s not easy today as most organizations use a mix of service providers for various aspects of their technology. But it’s necessary to effectively move up the compliance maturity model and being a trusted party in today’s world where trust is an essential asset.