Localizing Error-inducing Commits in CI Environments

Abstract

Nowadays, the practices around Continuous Integration (CI) have gained significant traction as project codebases are being impacted by a larger number of developers each time, and the need for a consistent and stable project remains a priority. In this regard, CI considers conducting a testing phase for each contribution made to a codebase. Despite of providing a higher level of confidence to the changes done, the testing phase can represent a time-consuming task. Also, it can become challenging to identify the contribution responsible for a failed test if multiple of them are being evaluated. Large IT organizations solve this by leveraging parallel jobs to validate each contribution, while still requiring to execute the test sequence on each of them. This thesis conceptualizes strategies that aid to identify contributions - i.e. commits made to a software repository - that led to a failed test. We call those error-inducing commits (EICs). To achieve this goal, existing code analysis techniques are explored and exploited to deduce the commits responsible for a test failure by associating the past code changes to the sections exercized by the test. In addition, this work provides a schema of properties related to the extend at which a tracing strategy discovers EIC candidates, which is used to categorize the conceived strategies. Furthermore, a tracing strategy is realized and its behavior is demonstrated in a set of common development scenarios. Towards the end of this work, suggestions are made for the further conception and realization of tracing strategies.