Unit testing

What are units?

Before we can start testing our units, we need to know what exactly a unit is.

This seems very straight forward and it usually is. In most of our cases units are classes. However a unit can also be a class method.

In any case it should be the smallest portion of code that can be tested.

Why should we test units?

As described in the chapter Ideal Application Structure it is very useful to separate code into small re-usable modules. In order for us to be 100% sure that our code will do exactly what we need it to do and to be able to guarantee that to others that potentially want to use our module, we need to test it.

We can never know how someone else will try to use our code, but we can be sure how our code will behave when it is used in a certain way. This behavior is what we need to guarantee. Of course it is possible to test the code by hand or write some code that uses our units in a way that we can use to verify the behavior, but that makes it difficult for a third party to check.

With unit tests, basically everyone that plans on using the code can also test the logic of the required units as the tests are included.

In conclusion unit tests should be used to guarantee that the API of our code can be used and will do exactly what it is supposed do.

What do we test?

This is a very important question we should ask ourselves every time we want to start writing tests. Of course, for analytical purposes, it would be a nice goal to have 100% code coverage.

Coverage basically means that that portion of the code is tested. 100% would mean that all of the lines of code that are written are also executed by at least 1 test.

While that sounds great, in a lot of cases it is really impractical and totally unnecessary. For example imagine we have a very standard data object:

<?php

namespace Scuti\Wallpaper;

class Wallpaper

{

    private $color;

    private $length;

    private $width;

    public function setColor(Color $color)

    {

        $this->color = $color;

    }

    public function setLength(int $length)

    {

        $this->length = $length;

    }

    public function setWidth(int $width)

    {

        $this->width = $width;

    }

    public function getColor(): Color

    {

        return $this->color;

    }

    public function getLength(): int

    {

        return $this->length;

    }

    public function getWidth(): int

    {

        return $this->width;

    }

}

This class has 4 public methods and we can test all of them, but is it really useful? The answer strongly depends on your own opinion, but it should not be a surprise to anyone that a mutator method overwrites the value of a class property and that the accessor method will do nothing more that return the value of the property.

Having said that, it would be totally acceptable not to test this class with unit tests and not have 100% code completion.

So what do we test using unit tests? Let's continue with our example by imagining we also have the following class in our module:

<?php

namespace Scuti\Wallpaper;

class RollCalculator

{

    private $wallpaper;

    private $wallWidth;

    private $wallHeight;

    public function __construct(Wallpaper $wallpaper, int $wallWidth, int $wallHeight)

    {

        $this->wallpaper = $wallpaper;

        $this->wallWidth = $wallWidth;

        $this->wallHeight = $wallHeight;

    }

    public function calculate(): int

    {

        $sheets = ceil($this->wallWidth / $this->wallpaper->getWidth());

        $totalLength = $sheets * $this->wallHeight;

        return (int)ceil($totalLength / $this->wallpaper->getLength());

    }

}

The class above can be used in our project to calculate how many rolls of wallpaper are required when someone wants to decorate a wall using wallpaper. Even though the business logic that this class contains consists out of only 3 lines, it is business logic nonetheless. If selling wallpaper is our business, we would want to make sure that this code produces the result we are looking for every single time.

So let's think about some test cases for this class, what are our variables?

It looks like the result depends strongly on the given input of the wall and of course the properties of the wallpaper. What is important about our wall is that we have to specify the width and height. What's important about the wallpaper is that the roll has a specific width and a length. Based on those values the outcome (number of rolls of wallpaper) is calculated.

The tweet below gathered quite a lot of fame and good laughs over time, but is actually a pretty good example of how we can test things.

In the tweet we can see that the first test case is probably the most common case. Many people will walk into a bar and order 1 beer. Next up are the extreme cases, this will probably not happen a lot, but it should still be possible to order 0 beers or 999999999 beers. Those are the positive test cases, but what if the code is used in way that it is not supposed to? Apparently it is also possible to order a lizard, -1 beers or a sfdeljknesv. Those are the negative test cases.

For our RollCalculator class let's start with a very basic case:

Wall width: 200

Wall height: 100

Wallpaper width: 100

Wallpaper length: 200

Calculation: ((200 / 100) * 100) / 200 = 1

In this case we test that, when using the variables above, the result will be 1 roll of wallpaper.

If we consider the test case above to be our "happy case", then now would be a good time to think about our extreme cases. The minimum case could be something like this:

Wall width: 1

Wall height: 1

Wallpaper width: 1

Wallpaper length: 1

Calculation: ((1 / 1) * 1) / 1 = 1

We still need 1 roll of wall paper in this case. Now let's try to come up with a test case for the maximum values. What we should keep in mind here is that there are limits to the numbers you can store in a specific variable type. The return type that is specified for our function is int and on 64bit operating systems running PHP that means that maximum value that can be returned is: 9223372036854775807, however since we have to deal with a float (because we are using the ceil() function) and calculations there are some precision errors, so for that reason we will choose a lower value (the maximum number that can be represented as a float without loss of precision).

Wall width: 1

Wall height: 9007199254740991

Wallpaper width: 1

Wallpaper length: 1

Calculation: ((1 / 1) * 9007199254740991) / 1 = 9007199254740991

That would be a lot of rolls of wallpaper. So based on our tests we have discovered that our logic loses precision and starts giving incorrect results after a certain value. It might be interesting to limit the value of some of the parameters, or to at least produce some kind of warning to the user that the result might be incorrect.

Given that in total we have 4 variables that have an influence on the result we should make test cases for each of the parameters or combinations of values for the parameters in such a way that we verify the behavior of the code.

The tweet above also has the cases: "Orders a lizard" and "Orders a sfdeljknesv". We should do the same in our tests. Our code requires the input to be of integer types, "lizard" however is a string and maybe "sfdeljknesv" is actually an object. When we use those values in our tests, we can verify what the logic does when the input that is given is of the wrong type. PHPunit for example will automatically convert PHP errors to Exceptions, which can then be used to assert the message of the error for example.

So we should not only test that our logic can be succesful in some cases, but it is equally important to test what happens when things do not go as planned.

Thinking about and implementing these kind of test cases will help prevent bugs and unexpected behavior. For example when we would pass a 0 as the wallpaper width a division by 0 warning will be generated. So we can refactor our logic to validate the input and throw an appropriate exception when validation fails or accept its limitations, but at least be aware of the flaw.

How do we test units?

The most common and convienent way to test our units is to use unit testing software. There are a few options to choose from, but most of the time the framework that is used in the project or module (if any) already includes a solution for unit testing. The 3 unit testing frameworks that seem to be widely used are:

1. PHPUnit
2. SimpleTest
3. atoum

Out of these 3 by far the most popular and most used is PHPUnit. In this chapter we will focus on PHPUnit, but the concepts of unit testing can be found in any of the frameworks.

The instructions of how to setup the framework and how everything works in detail, can of course be found on the respective websites of the frameworks. All of them can be installed using composer, but might require some extra configuration (location of the autoloader, execute some bootstrap, specify a different database, etc...).

Suites

A collection of test case files is called a test suite. Most projects have 1 big suite, but it is also possible to separate the suite into smaller suites. Doing so can be very useful, because it enables us to test only a part of the code. Especially with very big enterprise applications (or poorly written tests) it can take up quite some time when executing all the tests. Which would be a waste when the impact of the change is limited to only a very specific portion of the code.

Test cases

By test cases we mean the actual test cases. In PHPUnit the cases are defined as functions in files, that all contain 1 class that groups the cases together.

To keep our test cases specific enough to easily pin point where the code is failing, it is considered good practice to aim for having 1 assertion per test case. That might not always be possible, but we should make sure that our test cases do not verify too many things (especially multiple cases).

Assertions

To verify something inside a test case we use assertions. These assertions are available as functions of the base class that is extended to make a new test case file.

Assertions make the test pass or fail, so we could say that they are the most important part of the unit testing framework. Basically assertions compare the result of an operation (performed by our unit) with an expected value. When successful nothing happens and the test case will continue to be executed. When it is not successful a message is typically displayed and the next test cases are executed.

Test dependencies

In order to prevent writing and testing the construction (and/or initialization) of our classes in every test case, for example, it can be useful to declare that some test cases depend on other test cases.

Imagine the case where there are 2 test cases. The first test case verifies that our class can be constructed and the second test verifes that our class can perform some kind of action. If our class cannot be constructed into an object, it is pointless to try to make it perform some kind of action. Since that test case will fail because the first one fails, we can declare that the second test case depends on the first test case. Causing the second test case to be skipped when the first test case fails.

Example

<?php

namespace Scuti\Wallpaper;

use PHPUnit\Framework\TestCase;

class RollCalculatorTest extends TestCase

{

    public function testCanInstantiate()

    {

        $calculator = new RollCalculator(new Wallpaper, 1, 1);

        $this->assertInstanceOf(RollCalculator::class, $calculator);

    }

    /**

     * @depends testCanInstantiate

     */

    public function testHappyCase()

    {

        $wallpaper = new Wallpaper;

        $wallpaper->setLength(200);

        $wallpaper->setWidth(100);

        $calculator = new RollCalculator($wallpaper, 200, 100);

        $rolls = $calculator->calculate();

        $this->assertEquals(1, $rolls);

    }

}

The example above displays a test case file with 2 test cases.

In the first test case only instantiation of our class is tested.

After instantiating the class into an object there is an assertion that checks that the instance is of the type that we are expecting it to be. The assertion is not 100% necessary for this test case as the test case would fail when the class cannot be instantiated for some reason (a PHP error or an Exception will help do that). It is good practice to include at least 1 assertion, otherwise PHPUnit will flag our test, stating that it is a case without assertions.

In the second test case we can see that there is a dependency declared on the first test (so if the first test fails the second one is skipped). After instantiating and initializing our class we attempt to calculate the number of rolls of wallpaper that is necessary according to our logic and assert that our expected result of 1 roll is calculated.

Fixtures

Sometimes it is necessary to execute some code before or after every test case or test case class. That can be achieved by overriding the setUp() and the tearDown() functions if the code should be executed before and/or after each test case. For code that needs to be executed before and/or after each test case class, the setUpBeforeClass() and tearDownAfterClass() functions can be overridden.

For more information on how this works in PHPUnit, please check here.

Where do we test units?

Units are typically tested where they "live". So if our units are located in a module, then the right place to test them would be in the module as well. If our units are located in our project, then we should test them in our project.

We can use our local development environments to test our units, but of course it is also very convienent to test them on a CI environment (such as Jenkins, Bamboo, CircleCI, Travis, etc...).

As a rule of thumb, we should always make sure to test the units on an environment that is as similar as possible to the production environment as that is the environment where the units have to work. Having a development environment that is equal to the production environment is good practice in any case. Nowadays this can be achieved rather easily using tech like virtual machines or containers using Docker.

When do we test units?

There are several options when it comes to when to test units. For example it is quite common to have a nightly build system in place using a CI environment.

Another appropriate time to test our units would be when we are making changes to them. Right before releasing a new version of the units is always a good idea.

When we want to release our project that contains third party modules, it might be a good idea to run the modules' their unit tests to check if the version that was chosen works well in our (production) environment.

Also when choosing to use a new third party module, don't just assume it will work correctly, but verify it by executing the test suite that comes with it.