Combinatorial Testing

Survey (NIST Combinatorial Testing)

• A survey of combinatorial testing. Changhai Nie and Hareton Leung. ACM Comput. Surv. 2011.
• Community-Based, Collaborative Testing and Analysis. Atif Memon, Adam Porter, and Alan Sussman. In FSE/SDP 2010
• Combinatorial Software Testing, R. Kuhn, R. Kacker, Y. Lei, J. Hunter. IEEE Computer 2009.
• Combinatorial Testing. R. Bryce, Y. Lei, D.R. Kuhn, R. Kacker. Chap. 14, Handbook of Research on Software Engineering and Productivity Technologies: Implications of Globalization, 2009.
• Automated Combinatorial Test Methods: Beyond Pairwise Testing. D.R. Kuhn, R. Kacker, Y. Lei. Crosstalk, Journal of Defense Software Engineering, 2008.
• Practical Combinatorial Testing – Beyond Pairwise Testing, D.R. Kuhn Y. Lei, R. Kacker. IEEE IT Professional 2008.
• Combination Testing Strategies: a Survey. M. Grindal, J. Offutt, S. Andler. STVR 2005.

Debugging or Fault/Failure Characterization

• Pseudo-Exhaustive Testing for Software, D. Richard Kuhn and Vadim Okun, Software Engineering Workshop, 2006.
• Characterizing Failure-Causing Parameter Interactions by Adaptive Testing. Zhiqiang Zhang and Jian Zhang. ISSTA 2011.
• Adaptive Interaction Fault Location Based on Combinatorial Testing. Ziyuan Wang, Baowen Xu, Lin Chen, Lei Xu. QSIC 2010.
• Using symbolic evaluation to understand behavior in configurable software systems. Elnatan Reisner, Charles Song, Kin-Keung Ma, Jeffrey S. Foster, Adam Porter. ICSE 2010.
• Large Scale Distributed Testing for Fault Classification and Isolation. Sandro Fouché. Dissertation 2010.
• Incremental covering array failure characterization in large configuration spaces. Sandro Fouché, Myra B. Cohen, Adam A. Porter. ISSTA 2009
• Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces. Cemal Yilmaz, Myra B. Cohen, Adam A. Porter: TSE 2006.
• Main effects screening: a distributed continuous quality assurance process for monitoring performance degradation in evolving software systems. Cemal Yilmaz, Arvind S. Krishna, Atif M. Memon, Adam A. Porter, Douglas C. Schmidt, Aniruddha S. Gokhale, Balachandran Natarajan. ICSE 2005.
• An Evaluation of Combination Strategies for Test Case Selection,Mats Grindal, Birgitta LindstrÄom, Jeff Offutt, and Sten F. Andler, Emperical Software Eng 2006
• The Minimal Failure-Causing Schema of Combinatorial Testing, CHANGHAI NIE,HARETON LEUNG, ACM Transactions on Software Engineering and Methodology 2011
• Algorithms to locate errors using covering arrays, Conrado Martínez, Lucia Moura, Daniel Panario, Brett Stevens, LATIN'08

Testing Concurrent System

• A Combinatorial Testing Strategy for Concurrent Programs, Lei, Y.; Carver, R.; Kacker, R. N.; Kung, D,Software Testing Verification & Reliability 2007

Adaptive Test Generation

• Feedback Driven Adaptive Combinatorial Testing. Emine Dumlu, Cemal Yilmaz, Myra Cohen, and Adam Porter. ISSTA 2011.
• Finding Interaction Faults Adaptively using Distance-Based Strategies, R.C. Bryce, C.J. Colbourn, D.R. Kuhn, ICECCS 2010.
• Towards incremental adaptive covering arrays. Sandro Fouché, Myra B. Cohen, Adam A. Porter: ESEC/SIGSOFT FSE 2007

Event-Driven Testing

• GUI Interaction Testing: Incorporating Event Context. Xun Yuan, Myra B. Cohen. And Atif M. Memon. TSE 2011.
• Combinatorial Methods for Event Sequence Testing, D.R. Kuhn, J.M. Higdon, J.F. Lawrence, R.N. Kacker, Y. Lei. 8 Oct 2010 (submitted for publication).
• Testing Event Sequences
  • Towards Dynamic Adaptive Automated Test Generation for Graphical User Interfaces. Xun Yuan, Myra Cohen. And Atif M. Memon. TESTBEDS'09
  • Covering Array Sampling of Input Event Sequences for Automated GUI Testing. Xun Yuan, Myra Cohen. And Atif M. Memon. ASE 2007.
  • A Combinatorial Approach to Building Navigation Graphs for Dynamic Web Applications, Wenhua Wang, Jeff Lei, Sreedevi Sampath, Raghu Kacker, Rick Kuhn, James Lawrence, ICSM 2009.

Combinatorial Tests Generation (Algorithms)

• Constructing test suites for interaction testing, M.B.Cohen, C.J. Colbourn, P.B. Gibbons and W.B. Mugridge, ICSE 2003.
  • IPOG/IPOD: Efficient Test Generation for Multi-Way Software Testing, Y. Lei, R. Kacker, D. Kuhn, V. Okun, J. Lawrence, Journal of Software Testing, Verification, and Reliability,Sept. 2008
  • Interaction testing of highly-configurable systems in the presence of constraints, M.B. Cohen, M.B. Dwyer and J. Shi, ISSTA 2007

Tools

• Advanced Combinatorial Testing System (ACTS), NIST.