Continuous Compliance Testing

Motivation

This master thesis is motivated by the technical debt of security aspects, which arise due to increasing software and infrastructure complexity. Beside the challenges of growing pace in software development and the automation process of different stages like building, testing and deployment to provide reproducible artifacts continuously (c.f. DevOps), the utilized infrastructure gets also more considerable. To accomplish infrastructure requirements like maintainability, versioning, scalability and easy reproducibility, ``Infrastructure as Code" was created and can be used with configuration management systems like Puppet, Chef or Ansible. When integrated into a deployment pipeline, these tools allow fast and reliable preparation of required infrastructure in an automated manner, i.e. integrating those tools into the deployment pipeline of a certain software product.

Applying these techniques decreases the hardness of system configuration which unfortunately engenders decreasing awareness of correct and securely infrastructure building due to a lake of knowledge. If done wrong the adaption of these tools will easily lead to system misconfiguration and security vulnerabilities, since a whole infrastructure would change by changing one line of code in a configuration file. In general system misconfiguration does have serious impact on the system and software security as stated in the OWASP Top 10 report. Thus the automation of infrastructure should be take seriously.

To conduct these potential problems of security misconfiguration in the sense of compliance testing in form of Compliance as code comes into being , which allows systematic verification of infrastructure characteristics. As well as the creation of infrastructure (Infrastructure as code) these tests have to be integrated into %In an optimal setup, compliance testing is integrated into the process of continuous integration/delivery to automatically check the utilized infrastructure for compliance. Using tools like Chef InSpec, UpGuard or ServerSpec combined with available security guidelines like CIS Benchmark or BSI catalog of measures for hard- and software security, the overall process of compliance testing is very straight forward and well elaborated. A software vendor could easily check its used infrastructure against given standards, like BSI Benchmarks, for compliance.

Besides testing general standards against an infrastructure configuration, a software vendor should also be interested in verifying the customers infrastructure against its own specific product requirements. For example, if a software component requires special cryptographic lib to provided sufficient security, then it is necessary to check if this requirement is satisfied before deployment. Meaning the provided infrastructure by the customer needs to be tested for compliance as well. Therefore the vendor should able to define custom compliance requirements for single software artifacts. This option enlarges the focus of infrastructure compliance testing in such a way, that specific software requirements need to be conducted and verified as well.

Although it is easy to define certain compliance requirements on small scale, it gets more difficult when enlarging the scope towards enterprise systems. Hence a modular approach of defining, assigning and checking those compliance requirements - as it is already established for various standards - from a \emph{product point of view} get even more important.

In the current field of research the topic of Compliance as code'' and continuous compliance testing’’ is relatively new and not yet that present. Paper and Literature about extending the scope form DevOps to SecDevOps, DevSecOpsor DevOpsSec exists and all purpose that security should be embedded into the existing approaches of DevOps. To this extend first attempts facing ``Compliance as code’’ exists and enable security testing and compliance verification in an automated manner.

Tools like Chef InSpec, RedHat OpenScap, UpGuard and more, which exists and allow the definition as well as the execution of compliance tests. Unfortunately these tools just face the general definition of compliance rules, focusing the infrastructure mainly and no conjunction to a certain software product. As a consequence a software vendor can not model its software related compliance requirements and test those in an continuous manner using existing build pipelines.

Concept

To solve this challenge the thesis tries to answers two research questions and purposes a tool-based service to model compliance requirements on software architecture level, which should be based on existing tools like InSpec or OpenScap and common standards like SCAP\cite{waltermire2016scap}, which defines XCCDF\cite{waltermire2011xcddf} and OVAL\cite{MITRECorporation2011}. The two research questions (RQ) are as follows:

[RQ 1] What kind of infrastructure compliance requirements from a \emph{product point-of-view} are considerable?
[RQ 2] Which capabilities exists and which need to be created so that software vendors can easily model and automatically check if an infrastructure is compliant with the software specific requirement?

Beside answering these questions in general, the goal is to \emph{elaborate}, \emph{develop} and \emph{evaluate} a tool-supported proceeding for infrastructure compliance testing to detect and prevent security misconfiguration from a \emph{product point of view}. Therefore an existing tool like InSpec, which is an ‘‘open-source testing framework for infrastructure with a human and machine readable language for specifying compliance, security and policy requirements`` \cite{inspec} should be wrapped into a model-based approach, to allow definition of compliance rules on different levels of a software architecture. %Beside \emph{developing} the proposed tool, by which one should be able to easily model and equip a software with compliance rules on a component level, Additionally certain organizational requirements like roles and responsibilities have to be \emph{elaborated} and \emph{evaluated}. For example it needs to be clarified which stakeholders take an active part, who is responsible for defining compliance rules on a software architecture level or which information are essential for reporting. \clearpage To accomplish the intended tasks the overall process will be split in four fields, which each itself contribute to one of the three major named phases as depicted in Figure \ref{Fig:concept_proceeding}. The sub-tasks (1 - 4) and their outcomes are listed and shortly explained in the following.

Pre-Excution Process defines all necessary proceedings and requirements, which are needed to deduce compliance requirements for a specific software product. Additionally it clarifies the responsibilities of various stakeholders and give a blueprint how to issue specific compliance rules\footnote{A \texttt{compliance} rule is defined as a single compliance requirement. Several \texttt{compliance rules} are accumulated to a product specific \texttt{compliance set}. }.
Compliance Management Tool allows different stakeholders \newline (Productowner, Developer, CSO, etc.) to model the compliance requirements on different levels of architecture. The modeling process is represented by four levels (L1 - L4) each resting upon the previous level. These are proposed to be as follows:
- L1 - Define Compliance Rules Definition of general rules which are valid or should be reused for various software products.
- L2 - Model Components related compliance rules Modeling of available software components which are reused in various software products. Moreover linking of existing rules (L1) and/or definition of new, component specific rules which are not intended for reuse.
- L3 - Model Landscape related compliance rules Modeling of general software landscapes assembled of software components. Additionally linking of existing rules (L1), definition of new, landscape specific rules, prioritization of components rules (L2) in case of conflicts and declaration of rule impact.
- L4 - Model customer specific software project Modeling of concrete customer projects including specification of particular, customer related, informations using a CMDB or similar.Eventually additional definition of customer specific compliance requirements.
Execution Tooling represents the connection to a existing build pipeline maintained with Jenkins, Bamboo or similar. To allow easy integration and evaluation of the derived compliance set by use of a Continuous Integration (CI) system, a plugin should be proposed, if required.Post-Execution Process comprises reporting of testing results towards the stakeholders. The task in general is to define the information set and the way of reporting.