01.02 - Good Practices

Automated tests progress is nearly similar at the one of manual tests. Nevertheless, automation has specificities conditioning the way which automated tests are designed and realized.

Simplicity

An automated test must be as simple as possible: easy to write, easy to read and easy to understand.

The writing of complex tests increases the risk of error and so of false positive (test fail although the tested application isn't involved).

Functionnal Unicity

In the case of a manual acceptance test, it's common to follow long test procedures who are verifying a lot of functionality of the tested system in one scenario.

It's necessary due to the specifics constraints of manual acceptance tests: the acceptance tester is obliged using the tested application to set-up pre-requisites et verify post-conditions. Several system functions are tested like this with an only test.

Automated tests allow departing from those constraints, because the setting-up of pre-requisites and verifications can be done without the tested application. Each automated test can test only one function of the SUT.

This method has many advantages:

• • Tests are shorter and easier

  • The qualification of a function doesn't depend of the good functioning of another function

  • During a campaign test, it's possible to test only parts of function of the SUT

Independence

Steps of pre-requisites setting-up and environment cleaning-up must assure that test cases are strictly independant of each other. The execution of a test case should never depend of the previous test case's result.

This rule is essential for those reasons:

• • Campaign layout: at each campaign, it must be possible to choose which tests and in what sequence they must be executed.

  • Results interpretation: when tests depend on each other, it's more complicated to interpret causes of failure. The failed of a test can lead of the fail of the next test. So, it's impossible to know if they failed because of the first test's failure or because of real issues.

Static Data Test

An automated test must be able to be replayed as many times as necessary and obtain each time the same result.

To make it possible, the simplest solution is to use identic datas from an execution to another. This is particularly true for non-regression tests which are valid only if they are executed in strictly identical conditions. This is possible thanks to the setting-up and cleaning-up steps of the environement.

Dynamic Data Test

There are 2 exceptions to the previous rule. Some data tests can't be determined a priori because they depend of the context in which the test case is executed. Among those datas, there are dates and data generated by the application

1. DATES

All data set containing dates is subject to expiration. For example, a contract was active when tests were realized can expire after a certain period of time. This can lead to the failure of the tests who are using this data set.

To handle this problem, 2 strategies are possible:

• • The first one is to upgrade frequently tests cases. This solution requieres to set-up a follow-up procedure of data set's maintenance that could be expensive.

  • It's preferable to set-up a mechanism which allows to define the dates at the moment of the test execution, relatively to the day date (for example, the first Monday of the month, the first open-day of the year, the previous month...)

2. DATA GENERATED BY THE APPLICATION

A part of data generated by the tested application can't be determined a priori. It's for example identifiers or timestamps generated at the execution. It arrives that such data in output of a test case must be used in input of the next test case. So, it's necessary to stock them in order to use them later

Maintenance

One of the main brakes to automation tests reside in the need of maintaining them. That's why automated tests concern stable functions of the tested system which are little set to expand.

Despite those precautions, features of the SUT are going to lead of needing maintenance. So, we need to anticipate the features during the realization of the tests in order to minimize the maintenance charge.

1. DATA TEST CENTRALIZATION

Sometimes, because of the evolution of the data model for example, a test case must be revalued.

To minimize the maintenance charge, the data of a test must be centralized. Concretely, it means that the data of a test are replaced by parameters which values are saved in one or several parameters file(s).

So, in the case of a test case reevaluation, only this parameters file(s) are modified and not the totality of test cases.

2. COMMON PARTS POOLING

The common steps of several tests cases must be shared. So if a modification of the SUT affects a common step to several cases, corrections must be made at only one place. This implies :

• • To create shared modules from which tests cases will be built

  • To configure data of the shared parts in order to valorize them differently according to the tests cases.