Note that the latest version of Pex disables "[PexFactoryClass]" attribute for a class that includes factory method. For some sample code here, if it doesn't compile due to the factory class, do the following steps: (1). comment out "[PexFactoryClass]", (2) comment out "using Microsoft.Pex.Framework.Factories;", (3) make sure your (factory) class that contains factory methods is "public static". Note that you should have "static" before the (factory) class definition; otherwise, Pex will crash.
- Introduction to Pex
- Slides: Pex tutorial slides
- Overview video: Watch the Channel9 video of Pex
- Student group exercise and instructor interactive demo of Stack example
- Help Pex to generate non-primitive arguments (e.g., UIntStack objects as parameters of PUTs)
- Take a look at the factory method. Compare the two commented-out not-that-good factory methods with the existing best one. The comments for these three factory methods illustrate the rationales.
- Note that you may want to generalize as much as possible for your factory method as well! (see the test generalization principle below)
- Generalization and guidance in factory methods
- Test generalization (bottom-up way of producing PUTs)
- Student group exercise and instructor interactive demo of test generalization of a conventional unit test testFull
- Goal: expand the scope of input values (primitive-type values or object states) while still satisfying the assertion(s)
- Principle: think like an attacker: can you introduce a fault (such as adding some extra branch whose true branch will delete all the files in your hard drive to the code under test while still passing the test case? For example, if your test case exercises only the method argument x's value 2, then as an attacker, I will put in "if (x == 99) deleteAllFile();" and your test case cannot expose this seeded fault! This principle is similar to the mind set of mutation testing.
- Try your best to specify tightest specification or behavior for your code under test: for the tightest specification, if you have a fault in your code under test, your tightest specification will be violated; if your tightest specification is violated, you have a fault in your code under test
- Example: a multiply(int x, int y) method's tightest specification is "return/y == x" and another looser specification "((x > 0 && y > 0) implies (return > 0)) && (((x < 0 || y < 0) && !(x < 0 && y < 0)) implies (return > 0))" is not tight enough; that is, I can have a fault in the code under test while still satisfying the looser specification.
- Operational steps
- Turn each concrete primitive value (including even those occurring in assertions) to a primitive-type parameter of PUT
- Add Pex assumptions for PUT to allow the assertion(s) still to be satisfied
- Try to generalize the receiver object to a non-primitive-type parameter of PUT (and add Pex assumptions when needed) while still satisfying the assertion(s)
- Try to suppress the method calls (except for the last method call, being the method under test) as many as possible (and add Pex assumptions when needed) while still satisfying the assertion(s)
- Double check whether you can relax or remove some of the assumptions being added while still satisfying the assertion(s)
- Writing PUTs (top-down way of producing PUTs)
- Slides
- Regression testing
- Regression testing example: UIntStackRegressionTest.cs
- Dealing with environments
- Slides
- Input space partitioning with Pex
- Write each characteristic as an assumption
- For each block, write a clause
- Use || to connect the clauses for each block
- Write assertions to check the completeness and non-overlapping of these blocks
- Use Pex to generate all combinations of these blocks (as all paths through these assumptions)
- For each combination/path, think about whether you can write assertions
- Combine paths/combinations that share the same assertions
- Formulate coverage criteria as assumptions being inserted to the code under test but being semantic preserving
- Boundary value coverage
- Active clause coverage (MC/DC)