|
General debugging tips
- When you see that Pex doesn't cover a branch that you expect it to cover, you start your debugging process and try to create a micro-benchmark for this difficult branch
- It is good to see how Pex explores various paths. So you shall allow Pex to output all the tests (not filtering out any tests), each of which corresponds to one run/path
- To do that, add [PexMethod(TestEmissionFilter=PexTestEmissionFilter.All)] for the PUT (replacing the original [PexMethod] or add the enclosed attribute value assignment to be within the existing [PexMethod])
- Look at coverage report generated by Pex
- Take a look at what these bounds mean
- For the reported un-instrumented methods, walk through them
- All un-instrumented method issues should be dealt with as follows:
- If the method is related to multi-threading, or an environment interaction that Pex can clearly not symbolically analyze anyway, then "Ignore Uninstrumented method"
- Examples: Thread.Enter, Thread.Exit, Registry.GetKey, Network.Send
- When you ignore uninstrumented methods, then it's important that Pex doesn't have to explore their argument validation code, which means that you might have to create object factories that supply concrete values, or mimic the argument validation
- Example: the MailMessage constructor takes an email string. If MailMessage is not instrumented, Pex cannot get around the email-validation logic, and method always throws an exception. You will have to make sure that valid email strings are provided.
- Look at statements at the same block where some earlier statements are covered but later ones are not
- Cause: exceptions are thrown by the last covered statement in the block
- Look into the callee body of the last statement (if it contains a method call) to further investigate the coverage info in the callee method body
- Put a break point on the last statement and rerun all the generated tests and start debugging
- Then look at all the tests and their associated information to debug
- Especially look for the first (or many) tests that correspond to path bounds exceeded or duplicate paths
- Put a break point in one such test
- Click Tests-> Debug -> Tests in current context
- The test execution will stop on the break point and then you walk through the path by F11/F10
- Try to understand why Pex is not successful
- Creat a self-contained micro-benchmark to characterize the feature of the code related to the un-covered branch to cause difficulties for Pex
- Share your micro-benchmark with other students, Tao Xie, and/or Pex developers.
Useful tips
- Use PexSymbolicValue.IgnoreComputation to avoid exploration of a certain API. More details are here. It wraps a method into another method that won’t get monitored by Pex
- For A a = b.f(x);, you can put "A a = PexSymbolicValue.IgnoreComputation( () => b.f(x))();"
Known issues (not resolved in Pex) and ways to get around
- Interfaces
- when an argument type is an interface, if Pex cannt find any class in the references or the project code base under test to implement this interface, Pex will generate no objects. Such situations can happen inside a factory method
- Solutions:
- Option 1: add the classes implementing the interface to the references of the project under test
- Additional guidance: add [PexUseType(typeof(C))] where C is the class implementing the interface for the PUT
| |
|