Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests
Objective: Most of the existing reduction algorithms focus on decreasing the test suite’s size. Yet, the differences in execution costs among test cases are usually significant and it may take a lot of execution time to run a test suite consisting of a few long-running test cases. This paper presents and empirically evaluates cost-aware algorithms that can produce the representative sets with lower execution costs.
Method: We first use a cost-aware test case metric, called Irreplaceability, and its enhanced version, called EIrreplaceability, to evaluate the possibility that each test case can be replaced by others during test suite reduction. Furthermore, we construct a cost-aware framework that incorporates the concept of test irreplaceability into some well-known test suite reduction algorithms.
Results: The effectiveness of the cost-aware framework is evaluated via the subject programs and test suites collected from the Software-artifact Infrastructure Repository — frequently chosen benchmarks for experimentally evaluating test suite reduction methods. The empirical results reveal that the presented algorithms produce representative sets that normally incur a low cost to yield a high level of test coverage.
Conclusion: The presented techniques indeed enhance the capability of the traditional reduction algorithms to reduce the execution cost of a test suite. Especially for the additional Greedy algorithm, the presented techniques decrease the costs of the representative sets by 8.10 – 46.57%
@article{Lin2014,
author = {Chu-Ti Lin and Kai-Wei Tang and Gregory M. Kapfhammer},
journal = {Information and Software Technology},
number = {10},
title = {Test suite reduction methods that decrease regression testing costs
by identifying irreplaceable tests},
volume = {56},
year = {2014}
}