Your page title

Improving Automation in Developer Testing:

Achievements and Challenges

Tao Xie

Automated Software Engineering Group

North Carolina State University

Latest Talk Slides (PPT, 1.73M) (PDF, 810K)

Shorter Talk Slides on Test Oracles (PPT)

Developer testing, a common step in software development, involves generating sufficient test inputs and checking the behavior of the program under test during the execution of the test inputs. Complicated logics inside a method make generating appropriate arguments difficult. In testing object-oriented programs, generating method sequences to put the receiver object or argument objects into appropriate states further complicates test-input generation. Existing industrial and academic tools incorporate various techniques to address the challenges of generating sufficient test inputs. After the generated test inputs are executed, program crashes or uncaught exceptions can be used to indicate program problems, especially robustness problems. However, some program problems such as producing wrong program outputs do not crash the program. Since manually inspecting the outputs of a large number of generated test inputs is not feasible, existing industrial and academic tools incorporate various techniques to address the challenges of checking the behavior of the program under test during the execution of the test inputs. This talk will present an overview of achievements and challenges in improving automation in developer testing, especially on test-input generation (i.e., generating sufficient test inputs) and test oracles (i.e., checking the behavior of the program under test).

• Tao Xie. Improving Automation in Developer Testing: State of the Practice. North Carolina State University Department of Computer Science Technical report TR-2009-6, February 20, 2009. [PDF][BibTex]

Venues of Presentations:

• 03/28/2008: State of the Art in Testing and Analysis Day 2008
• 10/30/2007: Verify 2007
• 09/19/2007: TISQA 2007

Industrial Developer Testing Tools being reviewed in the talk:

• Agitar AgitatorOne
• Parasoft Jtest
• CodePro AnalytiX
• SilverMark Test Mentor
• Microsoft Research Pex (for .NET)
• Microsoft Research Spec Explorer (for .NET)
• Avaya Labs Research eXVantage

If you have relevant industiral developer testing tools to be reviewed or have comments on the slide content, please contact Tao Xie.

Tao Xie's Research on Automated Software Testing

• Test Generation
• Test Selection
• Test Abstraction
• Test Oracle Augmentation
• Regression Testing

You may be also interested in Tao Xie's presentations/tutorials on Mining Software Engineering Data.