We previously looked at Black Box Testing, a method of testing methods, classes or systems that relies solely on the design specification. Black box testing is incredibly useful for finding out which valid or invalid cases your code covers. It can also be designed before your code has been written, which makes it an excellent first step in the design process. On the other side of the spectrum is White Box Testing.
The main purpose of black box testing is to simulate what happens when a user knows how your system is supposed to work and tries to break it. They are looking for holes or gaps in implementation that produce errors. White box testing assumes that the implementation is correct and focuses on rigorously testing that implementation.
The white box testing is accomplished through code coverage. Code coverage is a general term that is meant to signify the percentage of lines of code covered by your tests. In order to achieve code coverage, however, you MUST know the code in great detail. This is the main way white box testing differs from black-box testing. As a result, white box testing is usually more thorough than black-box testing.
There are many techniques used to achieve code coverage. Each one is used for different purposes, but only when they are all used together is full 100% coverage possible.
Statement coverage is the easiest type of code coverage necessary in white box testing. To achieve statement coverage, every line of code must be executed at least once. In most cases, the set of black-box tests provided by the design team will cover 90-100% of the lines of code. The remaining lines of code would be covered, usually, by tests which cause errors (exceptions in Java). Occasionally, it is impossible to achieve 100% statement coverage. When that happens, just skip it.
Branch coverage is a stronger version of statement coverage. It aims to test every branch of code once. This includes:
Consider the following code (and flowchart):
int a = ... // Some integer input value
int b = ... // Some integer input value
if (a + b > 10) {
System.out.println("a + b is large");
}
if (a > 5) {
System.out.println("a is large");
} else {
System.out.println("a is small");
}
Here we have two if statements. The first is a simple if, but it still has 2 possibilities: a + b > 10, or a + b <=10. That means in total, there are 4 possibilities:
What we need is 2 cases out of these 4 possibilities that cover all branches (i.e. arrows) on the flowchart. If we choose Test 1 we cover both true branches. With our second test, we would then need to cover both false branches, which is accomplished with Test 4.
Questions:
An even stronger (and far more important) version of branch coverage is compound condition coverage. Let's say you have the following if condition:
if (!done && (colour.equals("blue") || value > 100)) {
It is not enough to simply use 2 test cases; one which skips the if (false) and one which enters (true). Instead, we are going to create 2^n True/False test cases for all simple expressions combinations within our compound condition. In this case, our simple conditions are:
Meaning, we will have 2^3, or 8 cases for this if statement alone. It is now easier to simply create a truth table to ensure we hit all of our possibilities. It is important to note: Sometimes, it will be impossible to satisfy all cases. When that happens ignore the impossible cases. (Question: which of these are "impossible" tests?)
You can see that this type of testing could quickly become unfeasible. It would be your judgement which of the tests can be omitted without compromising the quality of the testing. One approach, called partial compound condition coverage is to choose tests so that each expression is tested as either true or false. In our above case that would be:
Some of these expressions can be covered with the same test.
Questions: What would be the list of tests to achieve partial compound condition coverage of the following statements:
Much like if statements, loops have conditions, and as such there are multiple ways to test a loop. There are six major ways of testing loops:
Where N is the maximum number of times the loop can be executed.
Each of these cases is necessary in order to prove the value and correctness of the loop. Notice that, much like black box testing, we are testing near the limits. This is because that is where most of the errors occur.
Consider the following code. The readInts method takes in a scanner, an array of integers called the buffer, and an integer representing the length of the buffer. If we wanted to test the readInts method to ensure that the loop is executed the correct number of times, we would need the following test cases:
Questions:
Consider the following method:
public static void foo(int i, int n, int x, int[] v) {
while (i < n && v[i] < x) {
if (v[i] < 0) {
v[i] *= -1;
}
System.out.println(v[i]);
++i;
}
}
In order to test this method we need the ability to ensure that the loop will execute 0, 1, 2 and multiple times. Consider the code presented below.