Combinatorial testing (CT) is a well-known black-box test design approach. Based on the system’s specification, test inputs are generated to systematically detect errors in a system under test (SUT).
ROBUSTA: The degree to which systems can deal with invalid inputs and stressful environmental conditions, i.e. robustness, is an important characteristic of software systems. Robust is often implemented by if-statements that detect invalid inputs and exceptional control-flows that activate error-handlers to recover the system. Because the exceptional control-flow. When errors are detected, the normal control-flow ist left. These systems require different test inputs to prevent the so-called input masking effect. Therefore, this research project aims to improve the activities of modelling and generating test inputs to test the robustness of software systems.
But, the generation of effective test suites is not the only challenge in CT. After test execution, the test inputs of failing tests require further analysis to reveal failure-inducing combinations. Manual analysis is a tedious and cumbersome task, especially for SUTs with many input parameters and values.
COFFEe: Therefore, this research project aims to to fully automate the combinatorial (robustness) testing by deeply integrating testing lifecycle, the activities of test input modelling, test input generation, test method execution and fault characterization. COFFEe is a framework which integrates the aforementioned steps as a JUnit5 extension, which makes it easy to integrate it into software development and build processes.
coffee4j is the reference implementation of ROBUSTA and COFFEe and can be found accessed via https://coffee4j.github.io.
