Automated Functional Testing (Week 4)

Command Line

There are a number of ways to test command line applications. The simplest way is just to capture the output by redirecting it to a file and compare that file to a known "good" output. This style of testing is commonly known as "gold file" testing, the known good output is the "gold" file. This style of testing has drawbacks, though. Gold file tests can be brittle; small changes to the program can still cause an entire gold file test to fail. Additionally, the gold files themselves can be cumbersome to create and update.

We can also test program output in smaller chunks. We can do this manually, of course, but it's easier if we use a tool to make it easier. Let's take a look at how we might handle this manually and then we'll try a purpose-built tool.

Manual Testing Example

As an example, we will create a couple tests for a command line utility included by default on most systems.

BATS (Bash Automated Test System) Example

BATS is a tool for specifying and running tests on command line applications.

Web Browser

We can also functionally test web applications. Gold file tests are possible using images or raw HTML, but we can also automate the web browser itself in order to create more advanced tests that interact with and verify properties of individual HTML elements.

Puppeteer Examples

Puppeteer is a tool for automating a web browser. It is relatively recent and relies on a protocol that is currently only implemented in Google Chrome, but support is planned for all three major web browsers. As its name implies, Puppeteer allows us to interact with a web browser using code that can be assembled into scripts for repeatability.

We will use Puppeteer to verify some characteristics of some popular web sites as though we were developers testing them. Puppeteer is distributed as a JavaScript library and it can be installed using NPM.

You can find examples that use Puppeteer here:

• https://github.com/um-software-testing/reddit-puppeteer