Retrofitting Unit Tests for Parameterized Unit Testing

[Summary] [People] [Empirical Results]

PROJECT SUMMARY

Maintenance of software applications requires unit tests with high fault-detection capability. In practice, developers face two major challenges when manually writing unit tests with high fault-detection capability. First, developers may not be able to write test data that test all important behaviors of methods under test. Second, developers may unintentionally write redundant unit tests that exercise the same behavior of methods under test. To address these issues, developers can write Parameterized Unit Tests (PUTs), instead of unit tests without parameters, referred to as Conventional Unit Tests (CUTs). A major benefit of PUTs compared to CUTs is that developers do not need to provide test data in PUTs, since test data can be generated automatically using a test-generation tool. However, writing PUTs is more challenging than writing CUTs. For example, PUTs typically encode algebraic specifications, which are more abstract and general than sample-point behaviors encoded by CUTs. On the other hand, existing applications often include CUTs, which can be used to write PUTs with low effort. To exploit the benefits of PUTs in practice, we propose a novel approach to retrofit existing CUTs into PUTs. We conducted an empirical study on three real-world applications to show the benefits of retrofitting CUTs into PUTs. In our empirical study, we show that our approach retrofits 407 CUTs (4.6 KLOC) to 224 PUTs (4.0 KLOC). Along with achieving higher branch coverage (a maximum increase of 52% for one class under test and 10% for one application) than existing CUTs, our approach helps detect 19 new defects not detected by existing CUTs. Some of these defects are quite complex and difficult to detect using existing automatic test-generation tools alone.

The paper submitted to FASE 2011 can be found as an attachment below.

PEOPLE

Faculty

Tao Xie

Student

Madhuri R. Marri, Suresh Thummalapenta

Collaborators

Nikolai Tillmann, Jonathan de Halleux (Microsoft Research)

EMPIRICAL RESULTS

Subject 1: NUnit Framework V2.4.8

Tests for Namespace Generalized: nunit.util.dll

Subject 2: Data Structures and Algorithms v0.6

Tests for Namespace Generalized: All existing tests in the release package

Subject 3: QuickGraph v1.0

Evaluation Subjects

SPONSORS

Please contact Tao Xie for any additional information.