Checked Coverage for Test Suite Reduction – Is It Worth the Effort?

Roxane Koitz-Hristov*, Lukas Stracke, Franz Wotawa

*Corresponding author for this work

Research output: Contribution to conferencePaperpeer-review


As the size of software projects increases, their test suites usually grow accordingly. Test suite size, however, has a direct impact on the efficiency of software testing. Hence, test suite reduction (TSR) procedures aim at removing redundant test cases while maintaining the suites fault detection capabilities (FDC). This paper explores checked coverage as a coverage metric for TSR; checked coverage not only investigates if a part of code was executed but also if it was checked by a test oracle. Previously, this metric has been applied successfully as an indicator for oracle quality. To assess how suitable checked coverage is in comparison to traditional metrics, such as line or method coverage, we developed a TSR tool for Java programs. In an empirical evaluation, we performed TSR based on different reduction algorithms, coverage metrics, and open-source Java projects. Our study investigates both the efficiency of the TSR as well as effectiveness in regard to the FDC and size of the reduced test suites.
Original languageEnglish
Number of pages11
Publication statusPublished - 2022
Event3rd ACM/IEEE International Conference on Automation of Software Test: AST 2022 - Pittsburgh Pennsylvania
Duration: 17 May 202218 May 2022


Conference3rd ACM/IEEE International Conference on Automation of Software Test
Abbreviated titleAST 2022
CityPittsburgh Pennsylvania
Internet address


  • test suite reduction, checked coverage, experimental evaluation, software testing
  • test suite reduction
  • software testing
  • checked coverage
  • experimental evaluation

ASJC Scopus subject areas

  • Software
  • Control and Optimization
  • Safety, Risk, Reliability and Quality
  • Computer Science Applications

Cite this