Part III based on Selenium Tutorial written by Dr. Sarah Heckman

Part I: BDD

What is Behavior Driven Development (BDD)?

In the early 2000s a “test first” approach towards software development became a key component of Agile programming. This evolved into the more formal process of Test Driven Development (TDD) where a developer writes a test case (that should pass in the final product), then writes code until the test passes. Once the test passes the developer writes another (currently failing) test. The developer continues to alternate writing tests and writing functional code until the product is finished.

BDD extends TDD by focusing on understanding how the software is expected to behave prior to implementing the code, rather than simply rigging up a test suite. It also provides test results that are easier to interpret, regardless of the reader’s level of technical expertise.

What is Gherkin?

In BDD, the expected behavior of the program is written out using “ubiquitous language”, or a language and format agreed to by all members of the team¹. Gherkin is one of the most common formats used. Generally when we are talking about “Gherkin” we are referring to the format parsed by the cucumber framework. However, the ubiquitous language used by JBehave and other BDD frameworks is very similar.

Features

Gherkin breaks the product into “features” which are similar to User Stories². At the beginning of each feature is text explaining who the actors are, what the business value is, what business rules are applicable, and other information that is crucial to understanding the feature, but not captured in the individual scenarios described in the next paragraph^3,4. One format for this text is to specify that:

“As a/an <actor>

I want <description of feature>

So that <business value>”⁵

Given/When/Then

After the initial description come scenarios or scenario outlines. Scenarios and Scenario Outlines describe the different cases where a feature is used. Scenarios have specific inputs, while Scenario Outlines include variables which are then specified in a table. Scenario Outlines are useful where you want to use the same scenario with many different inputs (e.g. if you want to verify what happens when you buy 1, 3, and 10 cups of coffee, but anticipate having the same result in each instance). Scenarios and Scenario Outlines have three main parts

• 1. “Given” steps, which are the preconditions for the scenario: E.G. “Given the CoffeeMaker already has three Recipes”

  2. “When” steps specifying the primary actions to be taken in the scenario: E.G. “When I try to add another recipe’”,

  3. “Then” steps specifying the expected outcomes: E.G. “Then the recipe is not added”

A scenario can have more than one step of the same type. For example, a scenario may have more than one expected outcome. In this instance there is only one “Then” keyword, with the different preconditions separated by an “And” statement as shown below.

Then the correct error is thrown

And the recipe is not added

Note that ONLY the first word of a line is the keyword. If you use “and” in the middle of a line, it will not separate the line into separate steps.

You may have noticed that the previous scenario says “the correct error is thrown” rather than “A status message is printed to specify if the recipe was not successfully added”. In this part of the tutorial we will focus on creating acceptance tests that do NOT test the actual web interface of the system. We will discuss web testing in Part III.

Background Statements

If one or more preconditions are necessary for EVERY scenario and/or scenario outline in a feature file, it is possible to add a set of Background statements to the beginning of the feature file. For example, in iTrust we may want to be logged in as a certain physician, and we want the information of a very specific patient to be available.

Background:

Given I am logged in using MID 9XXXXXXXXX and password ***

And I have a patient who is ready to be released from the Hospital

These steps would be pre-pended to the steps in all other scenarios and scenario outlines in that feature file. For more information on background statements see: https://github.com/cucumber/cucumber/wiki/Background

Other Notes

Gherkin isn’t the only ubiquitous language used in BDD, nor is the “Given When Then” format described above uniquely used for BDD. When a team is following BDD, they are generally using Gherkin within a framework such as Cucumber which can only parse the information in the specification if it follows the right format. We will discuss Cucumber more in the next section

Part I Exercise

To practice what you have just learned, you will implement the the AddInventory user story. In this exercise you will convert the user story and black box test cases for AddInventory into a Gherkin Feature.

To complete this:

• • Use Gherkin to create a Feature description, Background (if necessary), and preliminary set of scenarios for the AddInventory user story. Verify that the scenarios you create AT LEAST cover the same scenarios outlined in the black box test cases here.

    • Note: The scenarios don’t have to be perfect at this point. Often it takes multiple iterations as you’ll see from the difference between the examples listed in the tutorial and the final AddRecipe feature file here.

Part II.A Cucumber

What is Cucumber?

Cucumber is a framework that “glues” Gherkin specifications (described in the previous section) to executable code that verifies whether the program is “behaving” as it should. Cucumber was originally developed to support BDD in Ruby, although there are now Cucumber-based frameworks for many different platforms. In this tutorial (and in iTrust) we will be using the java-based cucumber framework Cucumber-JVM. In this tutorial we will be using Cucumber to test an existing application rather than as part of BDD. Particularly when Cucumber is being used outside of BDD, the scenarios are often referred to as “tests”.

Installing Cucumber

For this tutorial:

Download CoffeeMaker_Cucumber_part2.zip. Once downloaded, import the project into Eclipse. Import the project as an “Existing Projects into Workspace” and select the archive zip file. Once the project has been imported into Eclipse, please add the project to your TomCat server. To do this, right click on the appropriate server and select “Add and Remove...”

For future projects, including iTrust:

When using Maven, the required libraries can be specified in the pom.xml and imported accordingly. If you aren’t using Maven or another build tool, you’ll need to download the appropriate files, most of which can be found here

For additional support: There are several Eclipse plugins designed to support Cucumber. I like this one: https://marketplace.eclipse.org/content/cucumber-jvm-eclipse-plugin which includes templates for feature files and a wizard for new Step Definition files (more on these later).

Writing Feature Files

Feature files for cucumber JVM are much like the Gherkin described above. The description of the feature does NOT need to follow the exact pattern listed above since it does not impact test execution. However, the scenarios themselves MUST match the specification to be properly executed. Feature files are stored together in the src/main/java or src/test/java directory and have the “.feature” extension. More information on how feature files must be formatted to be read by Cucumber is available here: https://cucumber.io/docs/reference. An example Feature File based on the AddRecipe user story can be found here.

Creating Step Definitions

Step Definitions are how a given statement in a feature file (e.g.“Given the CoffeeMaker already has 3 Recipes”) should be translated into executable code⁶. In Cucumber-JVM this is done by creating special java classes which contain methods corresponding with the statements in the feature files. Each method is annotated based on the type of statement (e.g. “@Given”) before the methods so the framework will know which methods correspond with which statements. For example:

The step definitions are stored in the same directory as the test runner, which will be described in the next section

Creating a Test Suite in Cucumber-JVM

To connect the framework together, you need to create a test runner. When leveraging JUnit-based tests such as the ones in this tutorial, the test runner is a java class similar to the one below:

As you can see, there is a CoffeeMaker that is shared between the classes, and an object called SharedRecipeData. The SharedRecipeData object contain information about recipes that need to be shared between steps.

PicoContainer leverages the Singleton pattern. In other words, it only instantiates ONE instance of whatever objects are passed in a given Scenario. This is good in some instances such as CoffeeMaker where we only WANT one instance of CoffeeMaker. However, when passing data you may want more than one instance (e.g. you may need to track separate MIDs for system users and patients in iTrust). Hence it can be helpful to have separate objects for shared information such as the SharedRecipeData object in the example above.

Part II Exercise

To practice what you have just learned, you will implement the the AddInventory user story and automate the testing of the testAddValidInventory acceptance test. Now create the step Definitions for the Gherkin specification you created earlier.

To complete this:

• • Download CoffeeMaker_Cucumber_part2.zip and import it into Eclipse, if you haven’t done so already.

  • Create a feature file called AddInventory.feature in src/test/resources/basic.

  • Go back and look at the feature file you created in the previous exercise. Copy the Feature description, any Background Statements you may have, and the first scenario or scenario outline to the new feature file, and save the file. See the AddRecipe.feature file for an example.

  • Create a class called InventoryStepDefs in src/test/java. Implement the step definitions for your initial scenario or scenario outline. See RecipeStepDefs.java for an example

  • Right-mouse-click on TestRunner.java and go to RunAs->JUnit Test

  • When the steps pass (green-bar), copy the next scenario into the feature file and create the necessary step definitions.

    • Note: you may find that you need to update the features, or even step definitions that you’ve implemented previously as you become more familiar with Gherkin, the framework, and the application.

    • You should set up the proper test data before each run, and your scenarios should green-bar, along with all other tests in the system.

    • You do not need to Javadoc your step definition class (but all other classes should have documentation)

    • When the server is running, the CoffeeMaker application can be accessed at http://localhost:8080/CoffeeMaker_Cucumber/

Part III: Webdrivers

Sometimes there are acceptance criteria that cannot be tested without opening up the user interface. For example, does the error for an incorrect input show up when you input the information in the textbox or when you click “submit”? To verify this functionality we will use a webdriver.

What is Selenium Webdriver?

Selenium is an open-source project with many components, of which we will only be using the WebDriver as part of our Step Definitions. Selenium WebDriver emulates how a user would interact with the browser. The WebDriver is actually an interface that is implemented as separate drivers for Chrome, IE, Firefox, etc. We will be using the HtmlUnit driver, which allows us to run the browser “headlessly” (it runs the browser without actually opening up a window) and which works well with Jenkins.

Installing Selenium

For this tutorial:

Download CoffeeMaker_Cucumber_part3.zip Once downloaded, import the project into Eclipse. Import the project as an “Existing Projects into Workspace” and select the archive zip file. Once the project has been imported into Eclipse, please add the project to your TomCat server. To do this, right click on the appropriate server and select “Add and Remove...”

For future projects, including iTrust:

In order to run Selenium tests, several jars are required. These jars are included in the lib file of the above CoffeeMaker project. To use Selenium on other projects, you will want to copy these jars. You will also need to add the jars to the build path of whatever new project you work on.

Using Selenium

WebDriver driver allows us to open a browser window and navigate, or drive, through the page to get to the desired page or perform the requested action. Throughout the tests, the WebDriver will act as the interface to the JSP page. For this reason, a single instance of the WebDriver will be passed through PicoContainer. The webdriver is wrapped in another class, CoffeeMakerDriver, which is used to ensure that the driver is closed at the end of execution. A few related methods values, such as the baseURL, are also contained in the CoffeeMakerDriver.

The selenium API has built in methods for identifying elements, like textboxes, links, and buttons, as well as methods for interacting with the elements through the driver object. An examples of an identifier and shown below. The full API can be found here:http://seleniumhq.github.io/selenium/docs/api/java/

The example above is fairly straightforward. However, if there isn’t a clear label in the application, you may end up with an identifier like the example below (or worse!) and quickly clutter up your test cases. These identifiers also are more likely to change. In the example shown, if the link became the child rather than the sibling of the ‘Add a Recipe’ header, the test would fail until the identifier is corrected.

Furthermore, methods provided in the Selenium API to interact with the identifiers also are not always representative of what a human would do. For example, while a human can simply visually scan a screen for a message, a Selenium test can only pull back elements that match a description provided and additional logic is necessary to determine if the list contains the value you are searching for.

PageObject pattern

To help resolve these issues, testing with Selenium and other webdrivers often leverages the PageObject design pattern. When using this pattern, the test developer creates a separate class for each page of the UI that encapsulates the functionality needed to test that page. In the CoffeeMaker tutorial, the page object classes extend the AbstractPageObject class. This allows them to inherit the driver global variable and setDriver() method.

In each PageObject class, the identifiers for the page are declared at the top of the class as private, global variables such as the one in the previous example. This makes the identifiers easy to find and fix. After that come the methods that one would use to manipulate the class. For example, in the CoffeeMaker tutorial, inputing new information into the Name field for a new recipe actually requires two steps, as shown in the method below:

In the example above nameBox an identifier specified earlier in the class:

The method AddRecipePage.inputNewName(webDriver, name) can then be called by whichever step definitions need to use the method, such as the one shown below:

The example above is somewhat simplistic. If all of the methods in your Page class are two lines or less, it may NOT make sense to use PageObject. Since the PageObject design pattern is likely to be helpful on iTrust and on larger projects you may work on in the future, we will use it in this tutorial. For more information on PageObjects see http://martinfowler.com/bliki/PageObject.html.

Part III Exercise

To practice what you have just learned, you will implement the the AddInventory user story, which will use the database for maintaining inventory and automate the testing of the testAddValidInventory acceptance test.

To complete this:

• • Download CoffeeMaker_Cucumber_part3.zip. and import it into Eclipse, if you haven’t done so already.

  • Copy the add_inventory.jsp file you completed as part of the JSP tutorial. If you have not completed the JSP tutorial, you will want to complete add_inventory.jsp so that you can test your solution.

  • Use Gherkin to create a Feature description, Background (if necessary), and preliminary set of scenarios for the AddInventoryWeb user story. This time, focus on aspects of the User Story and Black Box test cases (found here and here respectively) that you were unable to verify in the previous exercise.

  • Copy the Feature description, any Background Statements you may have, and the first scenario or scenario outline to the a feature file, and save the file as called AddInventoryWeb.feature in src/test/resources/web. See the AddRecipe.feature file for an example.

  • Create a class called InventoryWebStepDefs in src/test/java and implement the step definitions of the scenario or scenario outline. Create page object classes as necessary and appropriate. See RecipeWebStepDefs.java and the Page Object classes in the edu.ncsu.csc.coffeemaker.testUtils.web package of /src/test/java for examples.

  • Right-mouse-click on WebTestRunner.java and go to RunAs->JUnit Test

  • When the steps pass (green-bar), copy the next scenario into the feature file and create the necessary step definitions.

    • Note: you may find that you need to update the features, or even step definitions that you’ve implemented previously as you become more familiar with Gherkin, the framework, and the application.

    • Your tests should set up the proper test data before each run. In iTrust we will be able to do this through SQL. In the AddRecipe example provided, test data is staged using the WebDriver itself.

    • Your test should green-bar, along with all other unit tests in the system.

    • You do not need to Javadoc your new test class (but all other classes should have documentation)

    • When the server is running, the CoffeeMaker application can be accessed at http://localhost:8080/CoffeeMaker_Cucumber/

Submit

Export both part 2 and part 3 of your CoffeeMaker_Cucumber project and name it CoffeeMaker_Cucumber_unityid.zip (where unityid is your unity id). Submit CoffeeMaker_Cucumber_unityid.zip to the Homework 1 Part 2 submit locker.

^{1. https://en.wikipedia.org/wiki/Behavior-driven_development#Specification_as_a_ubiquitous_language}

^{2. https://en.wikipedia.org/wiki/Behavior-driven_development}

^{3. ibid}

^{4. https://github.com/cucumber/cucumber/wiki/Gherkin}

^{5. https://dannorth.net/introducing-bdd/}

^{6. https://cucumber.io/docs/reference#step-definitions}

^{7. While there are instances where switch statements are helpful, see https://watirmelon.com/2013/06/14/avoid-using-case-statements-in-your-cucumberspecflowjbehave-step-definitions/ for a list of reasons why they should be avoided if possible}

^{8. https://en.wikipedia.org/wiki/Dependency_injection}

^{9. http://picocontainer.com/introduction.html}