Probabilistic and Systematic Coverage of Consecutive Test-Method Pairs for Detecting Order-Dependent Flaky Tests

Paper: https://cs.gmu.edu/~winglam/publications/2021/WeiETAL21TACAS.pdf 

Dataset: https://cs.gmu.edu/~winglam/publications/2021/WeiETAL21TACAS.zip 

Implementation: https://github.com/Anjiang-Wei/TuscanSquare 

[New] Dataset confirmed with Maven Surefire: https://drive.google.com/file/d/13oGvRlGbg8upaK75q2VTZZwUsxWy4i2P

If you use any of this work, please cite our corresponding paper:

@inproceedings{WeiETAL21Tuscan,
    author      = "Anjiang Wei and Pu Yi and Tao Xie and Darko Marinov and Wing Lam",
    title       = "Probabilistic and Systematic Coverage of Consecutive Test-Method Pairs for Detecting Order-Dependent Flaky Tests",
    booktitle   = "TACAS 2021: Proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems",
    month       = "March",
    year = "2021",
    address = "Virtual Event",
}

We are grateful to Peter Taylor for a StackExchange post that led us to the concept of Tuscan squares. We thank Dragan Stevanovi´c, Wenyu Wang, and Zhengkai Wu for discussions about Tuscan squares and Reed Oei for comments on the paper draft. This work was partially supported by NSF grants CNS1564274, CNS-1646305, CCF-1763788, and CCF-1816615. We also acknowledge support for research on flaky tests from Facebook and Google.

Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).