Gregory M. Kapfhammer

Using Coverage Effectiveness to Evaluate Test Suite Prioritizations

Gregory M. Kapfhammer and Mary Lou Soffa. Using Coverage Effectiveness to Evaluate Test Suite Prioritizations. In the Proceedings of the ACM International Workshop on Empirical Assessment of Software Engineering Languages and Technologies, Atlanta, Georgia, November 2007.

Related Projects: Kanonizo and DIATOMS

Abstract

Regression test suite prioritization techniques reorder a test suite with the goal of ensuring that the reorganized test suite finds faults faster than the initial ordering. It is challenging to empirically evaluate the effectiveness of a new test case arrangement because existing metrics (i) require fault seeding or (ii) ignore test case costs. This paper presents a coverage effectiveness (CE) metric that (i) obviates the need to seed faults into the program under test and (ii) incorporates available data about test case execution times. A test suite is awarded a high CE value when it quickly covers the test requirements. It is possible to calculate coverage effectiveness regardless of the coverage criterion that is chosen to evaluate test case quality. The availability of an open source CE calculator enables future case studies and controlled experiments to use coverage effectiveness when evaluating different approaches to test suite prioritization.

[PDF] [PS] [BibTeX Entry] [Copyright Notice]

---

Links to this Page

• Complete News Listing last edited on 21 October 2009 at 5:22 pm by 141.195.226.29
• DIATOMS last edited on 26 August 2009 at 11:19 pm by 141.195.226.29
• Kanonizo last edited on 21 October 2009 at 5:06 pm by 141.195.226.29
• Research Papers last edited on 20 October 2009 at 5:41 pm by 141.195.226.29
• Evaluating Test Suite Prioritizations with Coverage Effectiveness last edited on 30 May 2008 at 2:00 pm by aldenv29.allegheny.edu
• Using Coverage Effectiveness to Evaluate Test Suite Prioritizations last edited on 21 May 2008 at 2:19 pm by aldenv29.allegheny.edu
• Computer Science 580, Junior Seminar, Spring 2008 last edited on 29 May 2008 at 11:54 am by aldenv29.allegheny.edu