Combinatorial testing is a sophisticated testing technique to reduce the amount of testing that would be necessary for exhaustive testing while still providing a similar value. It does so by specifying the coverage criterion, that all t combinations of input values of a system should be covered by a test set, with which most faults can be triggered. There are multiple generation algorithms for generating test sets for combinatorial testing, but to find the values, that actually caused the fault so-called fault characterization algorithms are used.

Although a multitude of algorithms is present, most combinatorial testing and fault characterization is still done manually. Therefore, work preceding this thesis introduced a framework for automating both, combinatorial testing and fault characterization [Bon18; BFL19; Kir19]. That work, however, violated certain framework characteristics, such as inversion of control and while the generation of test cases was automated, the execution was not entirely automated. As Continuous Integration becomes more important, it is, however, beneficial to automate as much of the testing process as possible.

Thus, this thesis extensively refactors the existing framework by reorganizing its modules and adding enhanced execution capabilities. Furthermore, additional functionality, such as callbacks during execution and functionally dependent parameters, is added. This work focuses on ensuring framework characteristics and enhancing extensibility. The thesis first analyzes the current state of the existing system and its flaws. The analysis results in requirements upon which the reorganized concept is based. As a proof of concept, the improvements are realized in Java and demonstrated both, with and without the testing framework JUnit. The implementation is evaluated and possible further areas of research are discussed.

Info

• • • Language: German, English
      • Supervisor: Konrad Fögen (E-Mail)
      • coffee4j on Github
      • ROBUSTA – Combinatorial Robustness Testing