TESTABLE REQUIREMENTS
Test Automation Best Practice #8: Write Testable Requirements
by Jackie King | Jun 20, 2018 | Best Practices , Test Automation Insights
Share article:
FacebookTwitterLinkedIn
Principles of Testable Requirements
A testable requirement describes a single function or behavior of an application in a way that makes it possible to develop tests to determine whether the requirement has been met. To be testable, a requirement must be clear, measurable, and complete, without any ambiguity.
For example, assume that you are planning to test a web shopping application. You are presented with the following requirement: “Easy-to-use search for available inventory.” Testing this requirement as written requires assumptions about what is meant by ambiguous terms such as “easy-to-use” and “available inventory.” To make requirements more testable, clarify ambiguous wording such as “fast,” “intuitive” or “user-friendly.”
Requirements shouldn’t contain implementation details such as “the search box will be located in the top right corner of the screen,” but otherwise should be measurable and complete. Consider the following example for a web shopping platform:
“When at least one matching item is found, display up to 20 matching inventory items, in a grid or list and using the sort order according to the user preference settings.”
This requirement provides details that lead to the creation of tests for boundary cases, such as no matching items, 1 or 2 matching items, and 19, 20 and 21 matching items. However, this requirement describes more than one function. It would be better practice to separate it into three separate requirements, as shown below:
When at least one matching item is found, display up to 20 matching inventory items
Display search results in a grid or list according to the user preference settings
Display search results in the sort order according to the user preference settings
The principle of one function per requirement increases agility. In theory, it would be possible to release the search function itself in one sprint, with the addition of the ability to choose a grid/list display or a sort order in subsequent sprints.
Testable requirements should not include the following:
Text that is irrelevant. Just as you can’t judge a book by the number of words, length by itself is not a sign of a testable requirement. Remove anything that doesn’t add to your understanding of the requirement.
A description of the problem rather than the function that solves it.
Implementation details. For implementation details such as font size, color, and placement, consider creating a set of standards that apply to the entire project rather than repeating the standards in each individual requirement.
Ambiguity. Specifications should be specific. Avoid subjective terms that can’t be measured, such as “usually.” Replace these with objective, measurable terms such as “80%.”
Solve Your Testing Challenges
Test management tool for QA & development
Download Free Trial
Approaches to Requirements
There are a variety of approaches to writing requirements: from traditional requirements documents to more agile approaches such as user stories, test-driven development (TDD), acceptance test-driven development (ATDD), and behavior-driven development(BDD). These approaches all benefit from following principles for testable requirements.
Icon User story
User stories
A user story is a requirement that is written as a goal, using language that avoids technical jargon and is meaningful to the end-user. User stories are brief and often follow the format: As a [user role] I want/need to [feature] so that [goal]. For example: “as a customer searching for a product, I want to choose whether to see the list of available products in a list or in a grid so that I can compare the available products.”
As the name implies, writing requirements as user stories puts the focus on the user or customer. By themselves, requirements expressed as user stories don’t have enough information to be testable. User stories should include acceptance criteria so that the team can know when the story is “done.” Read more about user stories at the Agile Alliance website.
Icon TDD
Test-driven development (TDD)
In TDD, requirements are written as unit tests. The unit tests are executed before any coding and should fail because the code they describe doesn’t exist yet. Then the code is written or refactored to make the test case pass, the test is executed again to ensure that it does pass, and then any necessary refactoring occurs.
This approach is sometimes called developer testing, both because this testing is performed by developers, but also due to where the testing occurs in the development cycle. However, testers have a valuable role to play in TDD. Testers can work with developers to create better unit tests, applying techniques such as boundary value analysis, equivalence partitioning, and risk analysis; and help ensure that necessary integration and workflow testing occurs. TDD tests are typically written in a tool such as Junit or VBUnit, and form an important part of the documentation of the application. Read more about TDD at the Agile Alliance website.
Icon ATDD
Acceptance test-driven development (ATDD)
In ATDD, user stories and their accompanying acceptance criteria become the tests that are used to demonstrate to a customer that the application works as intended. Acceptance tests are typically written in collaboration by a team of three, called the “three amigos”, that includes a user representative, a developer, and a tester. To make sure the tests are understandable by everyone on the team, they are written in “business domain” terms rather than technical terms.
The workflow in ATDD is similar to TDD: first, the user story is written, followed by the acceptance test. Then the user story is implemented, and the team repeats the acceptance test to confirm that it passes. Finally, any needed refactoring is done. It is possible for a team to practice both TDD and ATDD at the same time. For recommendations on writing good acceptance tests, refer to the article The ABCs of Acceptance Test Design by author Jeff Langr.
Behavior-driven development (BDD)
?One way to increase clarity in requirements is to write them as realistic examples rather than using abstract terms. This approach is referred to as specification by example (SBE) or behavior-driven development (BDD). BDD is similar to ATDD but uses a specific syntax called Gherkin. In BDD, user stories are supplemented with examples to create “scenarios.” The scenarios for a feature are stored together in a feature file that can serve as an executable specification.
BDD scenarios are written in using the GIVEN-WHEN-THEN syntax, as shown in the example below.
Feature: Search Results Display
As a customer searching for a product
I want to choose whether to see the list of
available products in a list or in a grid
So that I can compare the available products.
Scenario 1
Given: I perform a search for an inventory item
And: There are at least two items returned by my search
When: My preferences are set to list display
Then: I see a list of the items returned by my search
Scenario 2
Given: I perform a search for an inventory item
And: There are at least two items returned by my search
When: My preferences are set to grid display
Then: I see a grid of the items returned by my search
Scenario 3
Given: I perform a search for an inventory item
And: There are no items returned by my search
When: My preferences are set to list display
Then: I see a list of suggested alternatives
Tools for Testable Requirements
No special tools are necessary to create testable requirements. They can be documented in word processing files or even note cards. But tools can make the process more efficient. ATDD tests may be captured and automated using a tool such as FitNesse. For BDD there are also a variety of language-specific tools to write requirements in the Gherkin GIVEN-WHEN-THEN syntax and prepare them for automation, including the following:
Cucumber for Ruby
Jbehave for Java
SpecFlow for C#
Jasmine for JavaScript
Ranorex Studio’s robust tools and open API support all testing approaches, including TDD, ATTD, and BDD. To see how Ranorex Studio integrates with SpecFlow to automate BDD scenarios, read the article How to Use Ranorex Studio in Your BDD Process on the Ranorex blog.
Conclusion
Dozens of books have been published on the topic of writing effective software requirements, and this article presents just a brief overview of strategies for ensuring that your requirements are testable. But the most important strategy is to ensure that testers and user representatives are included early in the process of requirements definition. While testable requirements make it easier to automate your tests, the key goal is to ensure that the entire team shares a clear understanding of the requirements.