Testing

Testing in Go is very easy and well designed. All test codes are next to the source codes in every package and they facilitate both black-box and white-box testing at the developer's disposal.

General Rules of Thumbs

  1. Avoid global variables in the test script. It has direct influences to the entire package including the main source codes.
  2. Anything private in test script should start with the name "test", as in testYourNameHere.
  3. Anything public in test script should be the test suite themselves, as in func TestFunctionA(t *testing.T) {.
  4. If you can cover a certain portion of codes, heed the warning that the code smells. You should re-structure the source codes in a way that is testable and maintainable.
  5. You can access the private variables inside the package, like those private variables in the struct. Use it to your testing advantage.

Writing Test Script

Filenames

All test scripts should follow the source codes name with the ending _test before the extension. Example: sourceCodeName_test.go for sourceCodeName.go.


Table Driven Testing

If you read the Go documentation, you will immediately realize that they recommend you to implement a table-driven testing. Basically, the idea of such testing is that you always test against the Public API (most of the time) by feeding it a range of possible parameters. If you practice Effective Go and have a good basic use of the language, you can basically manipulate and covers all the source codes by this table-driven method.

The code pattern is basically as such:

func TestYourFunction(t *testing.T) {
        // formulate your test values
        <setup your common test values>

        // setup the table
        scenarios := []struct {
                inputName <type>
                ...
                outputName <type>
        } {
                {
                        inputName: <value>,
                        ...,
                        outputName: <value>,
                }, {
                        inputName: <value>,
                        ...,
                        outputName: <value>,
                }, {
                        ...
                        inputName: <value>,
                        ...,
                        outputName: <value>,
                }
        }

        // run test
        for caseNumber, scenario := range scenarios {
                // assemble and run test
                // report log
                // report result
        }
}

This way, you can just manipulate the test values without performing drastic changes over your function. The only downside however, is that if you have a detailed table of values, the function can be very long. Hence, this forces you to test smartly and effectively (rather than chucking in a lot of values).

Here is a full example of a test suite:

func TestNaClSymmetricEncrypt(t *testing.T) {
        commonPayload := []byte("En: A quick brown fox jumps over the lazy dog")
        largePayload := make([]byte, 16001)
        for i := 1; i <= 16000; i++ {
                largePayload[i] = 55
        }
        nonePayload := []byte("")

        commonKey := make([]byte, 32)
        _, err := rand.Read(commonKey)
        if err != nil {
                t.Errorf("FAIL: unable to prepare common key - %v", err)
                return
        }
        longKey := make([]byte, 64)
        _, err = rand.Read(longKey)
        if err != nil {
                t.Errorf("FAIL: unable to prepare long key - %v", err)
                return
        }
        shortKey := make([]byte, 16)
        _, err = rand.Read(shortKey)
        if err != nil {
                t.Errorf("FAIL: unable to prepare short key - %v", err)
                return
        }
        noneKey := []byte{}

        scenarios := []struct {
                inID      int
                inPayload *[]byte
                inKey     *[]byte
                inBadRand bool
                outError  bool
                outData   bool // we just test the out generated action, not algo
        }{
                {
                        inID:      0,
                        inBadRand: false,
                        inPayload: &commonPayload,
                        inKey:     &commonKey,
                        outError:  false,
                        outData:   true,
                }, {
                        inID:      1,
                        inBadRand: false,
                        inPayload: &nonePayload,
                        inKey:     &commonKey,
                        outError:  false,
                        outData:   true,
                }, {
                        inID:      2,
                        inBadRand: false,
                        inPayload: &largePayload,
                        inKey:     &commonKey,
                        outError:  true,
                        outData:   false,
                }, {
                        inID:      3,
                        inBadRand: false,
                        inPayload: &commonPayload,
                        inKey:     &longKey,
                        outError:  true,
                        outData:   false,
                }, {
                        inID:      4,
                        inBadRand: false,
                        inPayload: &commonPayload,
                        inKey:     &shortKey,
                        outError:  true,
                        outData:   false,
                }, {
                        inID:      5,
                        inBadRand: false,
                        inPayload: &commonPayload,
                        inKey:     &noneKey,
                        outError:  true,
                        outData:   false,
                }, {
                        inID:      6,
                        inBadRand: true,
                        inPayload: &commonPayload,
                        inKey:     &commonKey,
                        outError:  true,
                        outData:   false,
                },
        }

        cipher := NaCl{}
        oriReader := rand.Reader
        badReader := new(testRand)
        for _, scenario := range scenarios {
                rand.Reader = oriReader
                if scenario.inBadRand {
                        rand.Reader = badReader
                }
                data, err := cipher.SymmetricEncrypt(scenario.inPayload,
                        scenario.inKey)
                t.Logf(`
CASE %v
GIVEN:
payload=%v
key=%v

EXPECT:
error=%v

GOT:
data=%v
error=%v
`,
                        scenario.inID,
                        scenario.inPayload,
                        scenario.inKey,
                        scenario.outError,
                        data,
                        err)

                if (scenario.outError && err == nil) ||
                        (!scenario.outError && err != nil) {
                        t.Errorf("FAIL: unexpected error")
                }

                if (scenario.outData && data == nil) ||
                        (!scenario.outData && data != nil) {
                        t.Errorf("FAIL: unexpected encryption")
                }
        }
}

TIP:

  • When you define the value, always use key:value designation, not the original array style. This is because there will be time where you need to restructure your struct as your code grows (or linter complains about misaligned structure).
  • If you are using large test values, you would want to use to make good use of inID. This helps you to track your test case easily across a suite.
  • Always t.Logf to report the test case data instead of squeezing them inside t.Errorf.
  • Stick to back-box testing approach (against your Public API) as much as you can. Only in extreme condition, you would then use white-box testing against the private functions.
  • Be smart about manipulating private variables. They are very useful for reach-ability via mocking and isolation.

Run Test

Now that you have the test script ready, we can run the test. There are a few ways to do it.

Quick and Simple

The easiest way to do it is:

$ go test .

If you want to do it recursively:

$ go test ./...

This will run the test quick and easy way.


With Coverage

There are various way execute the test with coverage, notably using the -cover. However, to me, that statistic itself is useless. Fortunately, Go provides a "code coverage heat mapping" way which I adore and use till today.

Therefore, I will recommend only this way since:

  1. It tells you the test results
  2. It tells you the coverage percentage
  3. It tells you which codes are covered, not covered, or intensively covered.

At this point of writing (Go 1.12.1), It is still 2 lines commands. Here they are:

$ go test -coverprofile /tmp/gotest.out -v .
$ go tool cover html=/tmp/gotest.out -o /tmp/gotest.html
$ xdg-open /tmp/gotest.html        # or just use the browser to open the html result
  • The first command is to generate the coverage profile of your test suite and save the temporary output to /test/gotest.out.
  • The second command is to process the coverage profile data into a presentable html output, saved in /tmp/gotest.html.
  • The third command is optional, basically open the html in your browser.

If you do everything correctly, you get a heat map of your source codes, like this below, and continue to develop your test script strategically:

Go Testing - Example Coverage Heat Map

Race Conditions

Fortunately after Go 1.11 onward, go test facilitates race detector which is a very important tool and simplified a lot of test implementations. Although you need to write your test code to test concurrency implementation, consider using this race detector to simplify a bunch of tracking. All you need to do is to append the -race argument. Example:

$ go test -coverprofile /tmp/gotest.out -race .

OR

$ go test -cover -race .
$ go test -cover -race ./...
$ go test -cover -race "${HOME}/Documents/myproject/...

Disable Linter

In some cases, linter do fails to provide true positive results. Once in a while, you get a bunch of false positive reports. In those situations, you want the linter to skip the scan for particular line, section, or the entire package itself (source: https://github.com/golangci/golangci-lint#nolint).

To do that, simply append machine readable instruction into the scope. Example:

  • To disable by line (inline):
var bad_name int //nolint:golint,unused
  • To disable scope (append before the scope opening):
//nolint
func allIssuesInThisFunctionAreExcluded() *string {
  // ...
}

//nolint:govet
var (
  a int
  b int
)
  • To disable package (append before the package name):
//nolint:unparam
package pkg

NOTE:

  • Notice there is no space after the comment opening. This is machine readable comment. Leaving a space after the opening will turn the line into a pure comment which is not machine readable.

Bundle Together

Now that we have our desired tools (so far), it's time to bundle them together and make it a one line command. For me on Debian, I wrapped all the commands into a function and exported it out as a BASH function. Then I save this function in my ~/.bashrc. Example:

gotest() {
        open="false"
        arg="${1:-.}"
        if [ "$1" == "-r" ]; then
                arg="$2"
                open="true"
        fi
        go test -coverprofile /tmp/gotest.out -race -v "$arg" \
                | tee /tmp/gotest.log
        if [ $? != 0 ] ;then
               return 1
        fi
        2>&1 go tool cover \
                        -html=/tmp/gotest.out \
                        -o /tmp/gotest.html \
                        > /dev/null
        if [ "$open" == "true" ]; then
                xdg-open /tmp/gotest.html &> /dev/null
                xdg-open /tmp/gotest.log &> /dev/null
        fi
}
export -f gotest

Then, what I do is that I just have to call gotest to the place I want. Example:

$ gotest
$ gotest .
$ gotest ./...
$ gotest "${HOME}/Document/myproject/..."

If I want to see the detailed report, I just pass in the -r argument before the pathing:

$ gotest -r
$ gotest -r .
$ gotest -r ./...
$ gotest -r "${HOME}/Document/myproject/..."

That's all about testing in Go.