BDD

行為導向開發 (Behavior Driven Development, BDD)

2019/03/30 (新增內容)

2020/05/03 (新增連結 & 調整內容)

2020/06/09 (新增連結)

行為導向開發(Behavior Driven Development, BDD)是建構在Test Driven Development (TDD)之上，強調要以可測試的驗收條件 (Testable Acceptance Criteria)為撰寫需求的方式。BDD強調利用Given, When, Then描述不同情境下的需求，這個概念進一步可將需求轉換成可測試的驗收條件。如此一來，也更容易在系統開發前就已經準備好如何測試，也更容易將測試自動化(詳參: 3 Ways How Specification By Example and Gherkin Improve Collaboration)。BDD可以使用Gherkins(語言)來描述需求，使用Cucumber (或其他的工具)來轉換以Gherkins所寫的需求成為test case。

• Behaviour Driven Development in Agile — a Step by Step Guide to start

  • Phases of BDD in Agile

    • Discovery of Behaviour Driven Development in Agile

    • Formulation of BDD in Agile

    • Automation in Behaviour Driven Development in Agile

• Beyond BDD Best Practices

  • Define the actor — declare persona with attributes.

  • Be clear on the goal of the scenario — the scenario should have a concise title, a task that triggers an outcome, and an outcome.

  • Use concrete and realistic examples — but not too specific if no control over changing data.

  • One test per scenario — with exceptions such as when it takes too much time to reproduce the scenario.

  • Use application domain language — implementation is up to creativity. Use deep linking, API queries, SQL queries, whatever that works without sacrificing the writing style.

很多人會認為，開發的時間都來不及了，怎麼可能先準備好測試案例? 以BDD+TDD的概念，就是在寫需求的時候，就開始準備測試案例，也利用撰寫測試案例的過程來確認需求，這樣並不會花費更多時間，也可以節省誤會需求的浪費，另外，也會因為測試自動化，可以減少在測試上花的時間，也一併提昇開發的品質。

• 自動軟體測試、TDD 與 BDD

  • TDD 則是「先寫測試再開發程式」。沒有程式要怎麼寫測試呢？除了有些工具可以讓你寫測試時，一邊幫你產生空的類別與方法外（如 Eclipse 撰寫 Java 時就有類似的功能），一般來說也可以直接假設你已經撰寫好了程式，先揣摩如果已經寫好了這個程式該要如何使用。

  • BDD 則是比起 TDD 更進一步，除了在實作前先寫測試外，而在測試前還要先寫規格，但是這份規格並不是單純的敘述軟體的功能，而是這份規格，是一份「可以執行的規格」。

• 開發人員看測試之TDD和BDD

  • BDD是一種敏捷軟體開發的技術。它對TDD的理念進行了擴展，在TDD中側重點偏向開發，通過測試用例來規範約束開發者編寫出質量更高、bug更少的代碼。而BDD更加側重設計，其要求在設計測試用例的時候對系統進行定義，倡導使用通用的語言將系統的行為描述出來，將系統設計和測試用例結合起來，從而以此為驅動進行開發工作。

• 淺談TDD、BDD與ATDD軟體開發

• Software Architecture Rule of Thumb — Requirements!

  • Use Testable Acceptance Criteria

• 關於BDD/TDD的三大誤解

  • 誤解一：TDD是一種測試方法

    • TDD是一種「開發方法」，藉由先定義規格，再撰寫程式的方式來開發軟體。

  • 誤解二：TDD很神

  • 誤解三：TDD可治百病

• Behavior-driven development

• Behaviour-Driven Development

  • BDD Overview

• INTRODUCING BDD (by Dan North)

  • +Title: Customer withdraws cash+

    • As a customer,

    • I want to withdraw cash from an ATM,

    • so that I don’t have to wait in line at the bank.

  • +Scenario 1: Account is in credit+

    • Given the account is in credit

      • And the card is valid

      • And the dispenser contains cash

    • When the customer requests cash

    • Then ensure the account is debited

      • And ensure cash is dispensed

      • And ensure the card is returned

  • +Scenario 2: Account is overdrawn past the overdraft limit+

    • Given the account is overdrawn

      • And the card is valid

    • When the customer requests cash

    • Then ensure a rejection message is displayed

      • And ensure cash is not dispensed

      • And ensure the card is returned

• WHAT’S IN A STORY?

  • Story: Account Holder withdraws cash

    • As an Account Holder

    • I want to withdraw cash from an ATM

    • So that I can get money when the bank is closed

  • Scenario 1: Account has sufficient funds

    • Given the account balance is \$100

      • And the card is valid

      • And the machine contains enough money

    • When the Account Holder requests \$20

    • Then the ATM should dispense \$20

      • And the account balance should be \$80

      • And the card should be returned

  • Scenario 2: Account has insufficient funds

    • Given the account balance is \$10

      • And the card is valid

      • And the machine contains enough money

    • When the Account Holder requests \$20

    • Then the ATM should not dispense any money

      • And the ATM should say there are insufficient funds

      • And the account balance should be \$20

      • And the card should be returned

  • Scenario 3: Card has been disabled

    • Given the card is disabled

    • When the Account Holder requests \$20

    • Then the ATM should retain the card

      • And the ATM should say the card has been retained

Katalon

• Katalon 是個非常強大的測試工具，可以進行web UI test、web API test、Mobile test。也可以透過cucumber來落實BDD。

  • Manuals

  • Tutorials

  • Samples

BDD

在Katalon，測試是定義在TestCase裡，在TestCase裡可以執行Cucumber Features File，跟cucumber一樣，是利用Step Definitions去執行實際的測試 (詳參: Running Cucumber Features File 、Automation Testing with Cucumber BDD in Agile Teams)。

新增一個專案，New Sample Project，Project:Sample BDD Cucumber Tests Project

可以看到範例專案的Test Suites裡有一個有一個「Verify Operations」的suite。在這個suite裡其實是用到了 「Verify Divide」、「Verify Minus」、「Verify Multiply」、「Verify Add」等test case。在這些test case裡，使用到對應的feature，這些feature又對應了不同的step function (在include/scripts/groovy/operations裡)。

由於這些測試使用到webUI，測試的URL是: https://katalon-studio-samples.github.io/calculator/

我們先來試試看一個比較簡單的測試。

簡單的測試

先新增一個feature: Include/features/is_it_friday_yet.feature

Feature: Is it Friday yet?

Everybody wants to know when it's Friday

Scenario: Today is Tuesday

Given today is Tuesday

When I ask whether it's Friday yet

Then I should be told Nope

Scenario: Today is Friday

Given today is Friday

When I ask whether it's Friday yet

Then I should be told TGIF

新增一個step function:

scripts/groovy/com.sample.test/TestFriday.goovy

Katalon支援的是Groovy，語法跟Java類似

在step function裡，先定義given要執行的動作，內容必須跟feature內容對應，「(.*)」就是把內容變成參數，在下面的方法裡就會傳到「givenDay」裡，語法跟標準的cucumber語法有點不一樣 (目前還沒找到相關說明文件):

@Given("today is (.*)")

def Today_Is(String givenDay ) {

today = givenDay;

}

When也是同樣道理，在這裡呼叫is_it_Friday():

@When("I ask whether it's Friday yet")

def Whether_Is_It_Friday() {

actual_Answer=is_it_friday(today);

}

在Then的部分，利用assert來檢查真實的答案與預期的答案是否相同，當結果為真的時候，就是測試通過。

@Then("I should be told (.*)")

def I_verify_the_status_in_step(String expectedAnswer) {

assert actual_Answer == expectedAnswer;

}

scripts/groovy/com.sample.test/TestFriday.goovy

完整的程式:

package com.sample.test

import cucumber.api.java.en.Given

import cucumber.api.java.en.Then

import cucumber.api.java.en.When

class TestFriday {

/**

* The step definitions below match with Katalon sample Gherkin steps

*/

String today ="";

String actual_Answer="";

def is_it_friday(String today){

if (today.equals("Friday") ) {

return "TGIF";

} else {

return "Nope";

}

}

@Given("today is (.*)")

def Today_Is(String givenDay ) {

today = givenDay;

}

@When("I ask whether it's Friday yet")

def Whether_Is_It_Friday() {

actual_Answer=is_it_friday(today);

}

@Then("I should be told (.*)")

def I_verify_the_status_in_step(String expectedAnswer) {

assert actual_Answer == expectedAnswer;

}

}

執行測試，就可以看到測試的結果了。

當scenario很多的時候，可以不用寫很多個scenario，而是採用Scenario Outline，將內容參數化。

Include/features/is_it_friday_yet_again.feature

Feature: Is it Friday yet again?

Everybody wants to know when it's Friday

Scenario Outline: Today is or is not Friday

Given today is <day>

When I ask whether it's Friday yet

Then I should be told <answer>

Examples:

| day | answer |

| Friday | TGIF |

| Sunday | Nope |

| anything else! | Nope |

再進一步，假設我們要測試的是個Java類別，Friday，把以下檔案跟TestFriday.goovy放在同一個檔案夾下，這樣的測試就類似一般的單元測試:

Friday.java

package com.sample.test;

public class Friday {

String day="";

public Friday(String day){

this.day = day;

}

String isFriday(){

if (day.equals("Friday")){

return "TGIF";

}

else {

return "Nope";

}

}

}

修改一下TestFriday.goovy

scripts/groovy/com.sample.test/TestFriday.goovy

package com.sample.test

import cucumber.api.java.en.Given

import cucumber.api.java.en.Then

import cucumber.api.java.en.When

class TestFriday {

/**

* The step definitions below match with Katalon sample Gherkin steps

*/

String actual_Answer="";

Friday today;

@Given("today is (.*)")

def Today_Is(String givenDay ) {

today = new Friday(givenDay);

}

@When("I ask whether it's Friday yet")

def Whether_Is_It_Friday() {

actual_Answer=today.isFriday();

}

@Then("I should be told (.*)")

def I_verify_the_status_in_step(String expectedAnswer) {

assert actual_Answer == expectedAnswer;

}

}

目前在Katalon裡似乎指令還沒有辦法中文化，不過，內容是可以接受中文字:

Include/features/is_it_friday_yet_again.feature

@Friday

Feature: 是星期五嗎?

我們想知道今天是不是星期五

@Friday

Scenario: 今天是星期二

Given 今天是'星期二'

When 我問今天是不是星期五

Then 我應該被告知'不是星期五'

@Friday

Scenario: 今天是星期五

Given 今天是'星期五'

When 我問今天是不是星期五

Then 我應該被告知'快樂星期五'

scripts/groovy/com.sample.test/TestFriday.goovy

package com.sample.test

//import cucumber.api.java.en.And

import cucumber.api.java.en.Given

import cucumber.api.java.en.Then

import cucumber.api.java.en.When

class TestFriday {

/**

* The step definitions below match with Katalon sample Gherkin steps

*/

String actual_Answer="";

Friday today;

@Given("今天是'(.*)'")

def Today_Is(String givenDay ) {

today = new Friday(givenDay);

}

@When("我問今天是不是星期五")

def Whether_Is_It_Friday() {

actual_Answer=today.isFriday();

}

@Then("我應該被告知'(.*)'")

def I_verify_the_status_in_step(String expectedAnswer) {

assert actual_Answer == expectedAnswer;

}

}

scripts/groovy/com.sample.test/Friday.java

package com.sample.test;

public class Friday {

String day="";

public Friday(String day){

this.day = day;

}

String isFriday(){

if (day.equals("星期五")){

return "快樂星期五";

}

else {

return "不是星期五";

}

}

}

• BDD Testing Framework (Cucumber integration)

**作業**

練習使用Katalon，並且寫自己的feature及step function

***

看一下Katalon的範例，這個BDD範例結合了WebUI測試，也就是不只是可以直接測試程式碼也可以測試webUI:

• BDD Testing Framework (Cucumber integration)

• Include\features\BDD Cucumber Tests\Web UI Tests\Login Tests.feature

@tag

Feature: The test case verifies that a user can login with a valid account

Scenario Outline: Login successfully

Given The Login page is loaded successfully

When I login the system with valid "<username>" username and "<password>" password

Then The Dashboard Page is loaded successfully

Examples:

| username | password |

| demo@katalon.com | sPiHQ&YEa6ST`de+ |

• Include\scripts\groovy\login.LoginSteps.groovy

package login

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import static com.kms.katalon.core.testdata.TestDataFactory.findTestData

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.annotation.Keyword

import com.kms.katalon.core.checkpoint.Checkpoint

import com.kms.katalon.core.checkpoint.CheckpointFactory

import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords

import com.kms.katalon.core.model.FailureHandling

import com.kms.katalon.core.testcase.TestCase

import com.kms.katalon.core.testcase.TestCaseFactory

import com.kms.katalon.core.testdata.TestData

import com.kms.katalon.core.testdata.TestDataFactory

import com.kms.katalon.core.testobject.ObjectRepository

import com.kms.katalon.core.testobject.TestObject

import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords

import internal.GlobalVariable

import MobileBuiltInKeywords as Mobile

import WSBuiltInKeywords as WS

import WebUiBuiltInKeywords as WebUI

import org.openqa.selenium.WebElement

import org.openqa.selenium.WebDriver

import org.openqa.selenium.By

import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory

import com.kms.katalon.core.webui.driver.DriverFactory

import com.kms.katalon.core.testobject.RequestObject

import com.kms.katalon.core.testobject.ResponseObject

import com.kms.katalon.core.testobject.ConditionType

import com.kms.katalon.core.testobject.TestObjectProperty

import com.kms.katalon.core.mobile.helper.MobileElementCommonHelper

import com.kms.katalon.core.util.KeywordUtil

import com.kms.katalon.core.webui.exception.WebElementNotFoundException

import cucumber.api.java.en.And

import cucumber.api.java.en.Given

import cucumber.api.java.en.Then

import cucumber.api.java.en.When

class LoginSteps {

@Given('The Login page is loaded successfully')

def The_Login_page_is_loaded_successfully() {

WebUI.callTestCase(findTestCase('Web UI Tests/Advance Tests/Pages/Login Page/The Login page is loaded successfully'), [:], FailureHandling.STOP_ON_FAILURE)

}

@When('I login the system with valid "(.*)" username and "(.*)" password')

def I_login_the_system_with_valid_username_password(String username, String password) {

WebUI.callTestCase(findTestCase('Web UI Tests/Advance Tests/Pages/Login Page/Login with username and password'), [('username') : username

, ('password') : password], FailureHandling.STOP_ON_FAILURE)

}

@Then('The Dashboard Page is loaded successfully')

def I_verify_the_status_in_step() {

WebUI.callTestCase(findTestCase('Web UI Tests/Advance Tests/Pages/Dashboard Page/The Dashboard Page is loaded successfully'), [:],

FailureHandling.STOP_ON_FAILURE)

}

}

參考資料

• John Ferguson Smart (2014) BDD in Action / BDD in Action 2nd Ed.

• BDD and User Story Specification: Examples

• BDD in a Nutshell

• INTRODUCING BDD by Dan North (2006)

• Behavior Driven Development (wikipedia)

• No Code? No Problem — Writing Tests in Plain English

• Software Architecture Rule of Thumb — Requirements!

  • Use Testable Acceptance Criteria

• Postman

  • API Testing using Postman (with Chai-JS)

  • End-to-end testing of API endpoints with Postman and Postman BDD (with Chai-JS)

  • Getting Started with Postman BDD API Testing (Postman BDD)

• Java

  • JUnit Cucumber Example

  • Getting started with Cucumber in Java — A 10 minute tutorial

  • Cucumber Selenium Tutorial

• Java + Spring

  • Cucumber Spring Integration

  • Cucumber Tests in Spring Boot with Dependency Injection

  • Behaviour-Driven Development (BDD) with Cucumber, Junit and Spring Boot

• JavaScript

  • TDD / BDD with Javascript

    • Test Environment（Mocha、Jasmine、Jest、Karma）

    • Test structure (Mocha、Jasmine、Jest、Cucumber）

    • Assertion（Chai、Jasmine、Jest、Unexpected）

    • Creation, display, and watch test result（Mocha、Jasmine、Jest、Karma）

  • Mocha vs Jasmine, Chai, Sinon & Cucumber in 2019

    • Jasmine

    • Mocha + Chai + Sinon

  • Cucumber

    • 10 Minute Tutorial

    • Cucumber Testing Tutorials for Beginners

    • BDD in JavaScript: Getting Started with Cucumber and Gherkin (2017/6)

      • Your First Cucumber Test

      • More Advanced Cucumber.js Tricks

        • Asynchronous Step Definitions

        • Feature Background

        • Scenario Outlines

        • Data Tables

        • Hooks

        • Events

    • Getting started with Cucumber for JavaScript (2018/2)

  • Jest vs Mocha: Which Should You Choose?

    • Jest 包括在 create-react-app

    • If you have a large project with the need for flexibility and customization then Mocha is probably the choice for you. If you have a smaller project and don’t need the extra setup and configuration up front, Jest is probably the better option.

  • Mocha

    • 在瀏覽器中使用 Mocha 和 Chai 進行單元測試

    • describe / it / expect

  • Jasmine

    • describe / it / expect

  • Jest

    • describe / test / expect

• React Behavior Driven Development (BDD)

  • Jest Cucumber

  • Jest

    • Testing React Apps with Jest

      • If you are just getting started with React, we recommend using Create React App. It is ready to use and ships with Jest!

  • react-test-render

• TestCafe

  • TestCafe & CucumberJS

• End-to-End

  • Behaviour-Driven End-to-End Testing

    • cucumber + nightmare + chai

  • E2E TESTING WITH TESTCAFÉ AND CUCUMBER.JS

    • cucumber + testcafe

  • Automation Testing Using Cucumber Tool and Selenium – Selenium Tutorial #30 & Cucumber Selenium Tutorial: Cucumber Java Selenium WebDriver Integration

    • cucumber + Selenium WebDriver

• BDD by Teddy Chen

  • BDD（1）：詳盡的文件就是可用的軟體

  • BDD（2）：大家來吃小黃瓜之Cucumber運作原理

  • BDD（3）：在Eclipse執行Cucumber-JVM

  • BDD（4）：第一個Cucumber-JVM範例，上集

  • BDD（5）：第一個Cucumber-JVM範例，下集

  • BDD（6）：讓Step找到Step Definition

  • BDD（7）：使用Transform讓稅金同時支援5%和0.05表達方式

  • BDD（8）：實作第一個開發票Scenario

  • BDD（9）：DDD乍現

  • BDD（10）使用Scenario Outline指定多個範例（上）

  • BDD（11）使用Scenario Outline指定多個範例（下）

  • BDD（12） 第二個開發票Scenario：從BDD到TDD

  • BDD（13）新劇情找到新Bug

  • BDD（14）設計就是決定Form和Context的界線

  • BDD（15）用單元測試驗證類別行為

  • BDD（16）SBE、BDD、ATDD、TDD有何不同？

  • BDD（17）Given-When-Then怎麼寫？

  • BDD（18）這是一個End-To-End的Scenario嗎？

  • BDD（19）幫開發票功能加上使用者介面的三種方法

  • BDD（20）使用者介面細節放哪裡

  • BDD（21）從測試金字塔看BDD的自動化驗收測試

  • BDD（22）狀態與行為測試

  • BDD（23）從規格、程式和測試思考BDD要解決什麼問題

Cucumber with Javascript / react

• 10 Minute Tutorial (Cucumber.js / javascript)

npm install cucumber

features\is_it_Friday_yet.feature

Feature: Is it Friday yet?

Everybody wants to know when it's Friday

Scenario Outline: Today is or is not Friday

Given today is "<day>"

When I ask whether it's Friday yet

Then I should be told "<answer>"

Examples:

| day | answer |

| Friday | TGIF |

| Sunday | Nope |

| anything else! | Nope |

中文版

# language: zh-TW

功能: 是星期五嗎?

我們想知道今天是不是星期五

場景大綱: 今天是或不是星期五

假如 今天是 "<day>"

當 我問今天是不是星期五

那麼 我應該被告知 "<answer>"

例子:

| day | answer |

| 5 | 快樂星期五 |

| 0 | 不是星期五 |

| 4 | 不是星期五 |

| 1 | 不是星期五 |

利用step definition來測試程式碼。如: features\step_definitions\stepdef.js

const assert = require('assert');

const { Given, When, Then } = require('cucumber');

function isItFriday(today) {

if (today === "5") {

return "快樂星期五";

} else {

return "不是星期五";

}

}

Given('今天是 {string}', function (givenDay) {

this.today = givenDay;

});

When('我問今天是不是星期五', function () {

this.actualAnswer = isItFriday(this.today);

});

Then('我應該被告知 {string}', function (expectedAnswer) {

assert.equal(this.actualAnswer, expectedAnswer);

});

• Jest-cucumber (詳參: React Behavior Driven Development (BDD))

如果還沒安裝過typescript，也可以不使用typescript

npm install typescript

利用create-react-app產生範例並開啟typescript

npx create-react-app . --typescript

npm install jest-cucumber

src/features/is_it_friday_yet.feature

Feature: Is it Friday yet?

Everybody wants to know when it's Friday

Scenario Outline: Today is or is not Friday

Given today is "<day>"

When I ask whether it's Friday yet

Then I should be told "<answer>"

Examples:

| day | answer |

| Friday | TGIF |

| Sunday | Nope |

| anything else! | Nope |

src/features/step_definitions/is_it_friday_yet.steps.test.js

//jest-cucumber in javascript

import { defineFeature, loadFeature } from 'jest-cucumber';

import isItFriday from '../../utils/friday';

const feature = loadFeature('./src/features/is_it_friday_yet.feature');

defineFeature(feature, test => {

test('Today is or is not Friday', ({ given, when, then }) => {

let today;

let actualAnswer;

//regular expressions for parameter

given(/^today is (.*)$/, givenDay=>{

today = givenDay;

});

when('I ask whether it\'s Friday yet', () => {

actualAnswer = isItFriday(today);

});

//regular expressions for parameter

then(/^I should be told (.*)$/, expectedAnswer => {

expect(actualAnswer===expectedAnswer);

});

});

});

也可以寫jest的測試，如:src/utils/friday.test.js，也有人認為這樣的寫法(describe/test/expect)就是BDD。不過，應該是比較像cucumber的step definition，適合程式設計師閱讀，一般使用者應該不太容易看得懂這樣的內容。

import isItFriday from './friday';

describe('Today is or is not Friday',() => {

test('Sunday is "Nope"', () => {

expect(isItFriday('Sunday')==='Nope');

});

test('Friday is "TGIF"', () => {

expect(isItFriday('Friday')==='TGIF');

});

});

describe.each`

day | answer

${'Friday'} | ${'TGIF'}

${'Sunday'} | ${'Nope'}

${'anything else'} | ${'Nope'}

`('$day + $answer', ({day,answer}) => {

test(`Sunday is ${answer}`, () => {

expect(isItFriday(day)===answer);

});

});

src/utils/friday.js

export default (today) => {

if (today === "Friday") {

return "TGIF";

} else {

return "Nope";

}};