Generalization and guidance in factory methods

    • Take a look at the factory method. Compare the two commented-out not-that-good factory methods with the existing best one. The comments for these three factory methods illustrate the rationales.
    • Note that you may want to generalize as much as possible for your factory method as well! (see the test generalization principle below)
    • Below are more notes on making tradeoffs between generalization and guidance in factory methods

When you create your factory methods, you may consider two competing factors:

(1). If you don't give Pex enough guidance or enough constraints on narrowing down Pex's search space, Pex may not be able to generate test inputs that you want.

(2). If you don't generalize enough in your PUT or factory methods, you may miss some faults with the generated test inputs.

Take a look at the three factory methods that I created earlier:

http://7125570499225398210-a-1802744773732722657-s-sites.googlegroups.com/site/teachpex/Home/samples/UIntStackFactory.cs?attredirects=0

Note that if Pex or any other test generation tool is perfect in test generation (which is not possible all the time), then the following line in the third (best) example factory method can and should be removed.

PexAssume.IsTrue(elems.Length <= (uIntStack.MaxSize() + 1));

That is because if a fault can be found by only invoking on a stack with pushes of (uIntStack.MaxSize() + 2) elements, then the above assumption prevents the generation of a test input for detecting such a fault.

However, here we need to have such an assumption for the factory method in order to allow Pex to be guided to explore elements.

So for the second part of exercise Generalize PUTs from conventional unit tests and write good factory methods for non-primitive parameters, I would suggest you to first create the most generalized factory method, which doesn't constrain any valid input values. Then if Pex cannot generate good test inputs for covering the behavior in the PUT, you can add more assumptions iteratively to your factory method to guide Pex by narrowing down the search space till Pex can generate good enough test inputs.

Note that in general, you shouldn't compromise the generalization extent in your PUT (the above tradeoffs are suggested only in the context of factory methods). Your PUT should reflect exactly what your program under test is supposed to behave, serving the purpose of specifications.

PUT allows the separation of behavior specification and test generation.

Factory methods are created to help test generation. There, it is fine for you to make tradeoffs to reduce generalization and allow Pex to generate useful test inputs. But it is generally not a good practice for you to add additional assumptions (unnecessary from the point of view of behavior specification) in your PUT to help test generation.

For the above example factory methods for UIntStack, if the third one cannot guide Pex to generate sufficient test inputs, you may have to fall back to the second or even first factory method (less generalized but giving more guidance to Pex) to allow Pex to generate sufficient test inputs. This case may occur in the factory method that you are expected to generate for the ShoppingCart class in exercise Generalize PUTs from conventional unit tests and write good factory methods for non-primitive parameters.