Abstract

Regression testing in software development aims for uncovering bugs introduced to a system by changes. Depending on size and complexity of the system numerous regression tests might require a considerable amount of time. However, this time is not always available. This can be due to organizational reasons, e.g., in a situation where developers have only a short time span for testing, or technical reasons, e.g., the timespan for regression testing within a Continuous Integration environment may be limited in order to keep it feasible.

Two independent approaches to optimize regression testing for situations with limited time have been proposed in literature: test prioritization and test selection. A prioritization of tests arranges the tests, such that tests, that are more likely to fail or whose failure is critical, are executed first. Thus, regression testing can provide essential feedback early and test the important parts, when time is limited. Test selection, on the other hand, is about reasoning, which tests can be excluded with respect to this change in order to improve efficiency while preserving the safety, that no bugs are missed due to this exclusion. The efficiency of this approach varies depending on the situation. There are situations, where it cannot be avoided, that changes affecting the whole system will require a majority of the tests to be performed.

UML component diagram of the software architecture.

The vision for this master thesis is to build a framework for various selection and prioritization techniques, that can easily be integrated in an Integrated Development Environment or a Continuous Integration environment. The framework will aim for the Java programming language, build upon the well elaborated JUnit testing framework, and should be integrated into existing build environments, like Maven. A benefit of such a framework is, the possibility to implement, use, and compare test prioritization and selection approaches. Research challenges are, providing the different prioritization and selection approaches with the required data, integrating these approaches into the framework, and adapting the existing testing mechanisms, such that tests can can be excluded and run in a specific order.

Resources

• Thesis report
• Slides of final talk
• Prototype demonstration video