moca

Facilitating Safe and Sufficient Testing of Environment-Dependent Software

[Summary] [People] [Sponsors]

PROJECT SUMMARY

Unit testing ensures high quality of software at an early stage of a software development life cycle. A commonly used testing criterion in unit testing is structural coverage of the code under test. Manually writing test inputs to achieve high structural coverage of the code is a cumbersome task. To reduce the laborious manual effort, developers can use automatic test generation tools that automatically generate test inputs to achieve high structural coverage. However, environment-dependent software, such as embedded software, heavily interacts with environments such as hardware or operating systems. Applying an automatic test generation tool to test such software faces two major challenges: (1) generating environment states to achieve effective testing of such software, such as achieving high structural coverage, and (2) causing no harm to the safety of the environment such as unintentionally deleting a file when interacting with the file-system environment. To address these two challenges, we present the first systematic approach, called Moca, to assist developers in building a mock object that can help developers in effectively testing environment-dependent software. In our approach, mock objects simulate the behavior of the environment. We conduct an empirical study to evaluate our approach on a real-world application. In our study, we build a mock object based on Moca to simulate the file-system environment and show that the mock object can help achieve effective testing of environment-dependent software while ensuring safety of the environment.

The paper submitted to EMSOFT 09 can be found as an attachment below.

PEOPLE

Faculty

Tao Xie (Principal Investigator)

Graduate Students

Madhuri R Marri (MS Student)

Collaborators

Nikolai Tillmann, Jonathan de Halleux (Microsoft Research)

EMPIRICAL RESULTS

Results of our study show that Moca can assist developers in building a mock object that can help in achieving effective testing, without any false alarms and therefore being sufficient when compared to a naive implementation, while being less complex and thus incurring less implementation and maintenance effort than a sophisticated implementation.

The details of our results can be found below as an attachment (Results.xls).

SPONSORS