Automated Fault Localization for Combinatorial Testing

Combinatorial testing is a model-based testing technique. It guarantees that at least one test case covers any combination between values of t or less parameters. Most combinatorial testing is done manually, but there are also attempts to integrate it into automated testing frameworks. In light of the increased movement towards continuous integration/delivery this is extremely important. After the execution of a combinatorial test suite, the tester is left with the knowledge of which test cases failed. To correct the defect which caused the failure(s), fault localization for combinatorial testing attempts to find the sub-combinations responsible for failures. Numerous fault localization algorithms have already been developed in this active field of research. Some researchers already build a tool to support manual, but not automated, fault localization. The absence of such automation hinders widespread adaption and faster development of new algorithms. This thesis presents an approach to automated fault localization. After a brief overview of the current state of combinatorial testing research, a general concept is presented. This includes requirements every fault localization automation framework must fulfill. From these requirements a general architecture is derived. As a proof of concept, this thesis presents a Java implementation of said framework. Finally, the implementation is evaluated and possible future areas of research are identified.

Click here for the final thesis (PDF).