Testing Practice

Note: some of these answers may be subject to discussion -- we can discuss them on the forum, and then I sometimes (if they’re really important!) reflect that discussion back on here. I would resist printing these solutions until they’ve stabilised, a couple of weeks after they’re posted.

Goals of Testing

[x] ensuring our code correctly implements our specifications

[ ] ensuring our code has zero bugs

[x] trying to find defects in our implementation

[x] trying to localize these defects within our system


Testing is a balance between time and resources invested to find a bug versus the risk of a certain defect if allowed to leak into production code. Trying to remove all bugs from the system is idealistic, so the focus is on resolving the impactful ones. If we are unable to completely fix some bugs, we want to make sure that their impact is limited. Not only should our system be usable, however, it should correctly implement the specification (i.e. do what we intended it to do.)

Why do anything other than unit test?

[ ] we need to be able to test the individual pieces of our system

[x] we need to be able to test the integration of the pieces of our system

[x] we need to be able to simulate how users interact with our system

[x] we need to be able to simulate how our system functions in real scenarios


Unit tests do a great job of isolating bugs and making sure each piece works individually, but cannot truly model the experience of interacting with the system from the top down.

Mocking Components:

When do you think it's a good idea to use mock objects/components?

[ ] Never - they're overused, and then you have to test your mock objects, and they become a problem in themselves

[ ] For each of your dependencies, regardless of size -- they are great to facilitate testing

[X] For large components that are unreliable

[X] For any unreliable component (If the component is small, a very very simple mock might be enough -- akin to a stub that returns some valid data)

[X] For any component that returns random or non-deterministic responses -- for when you want to predict the behaviour under test circumstances

[ ] In production code, if the mock object provides enough of the functionality of the real thing

[X] For when you want to isolate behaviour during testing of a particular component

Writing Tests

Consider the HouseHunter methods:

//MODIFIES: this

//EFFECTS: adds the house to the list of houses to visit

public void addInterestingHouse(House h){ }


//EFFECTS: returns the most recently added house

// and takes it off the list of houses to visit

public House getNextHouseToVisit(){ return new House(); }


You have implemented this test method:

//TODO: make sure two houses are successfully added

@Test

public void testAddTwoHouses(){

// setup the test

HouseHunter hh = new HouseHunter();

House h1 = new House();

House h2 = new House();


// call the method to test

hh.addInterestingHouse(h1);

hh.addInterestingHouse(h2);


// check the outcome

House gh1 = hh.getNextHouseToVisit();

House gh2 = hh.getNextHouseToVisit();


if (gh1 == null) { fail(); }

if (gh2 == null) { fail(); }


assertEquals(hh.getNextHouseToVisit(), h1);

assertEquals(hh.getNextHouseToVisit(), h1);

assertEquals(hh.getNextHouseToVisit(), null);

assertEquals(hh.getNextHouseToVisit(), h2);

assertEquals(hh.getNextHouseToVisit(), h1);

}

Your test is just not working and you suspect you have some unnecessary parts. Below, select the lines of code that should stay

[X] hh.addInterestingHouse(h1)

[X] hh.addInterestingHouse(h2)

[ ] House gh1 = hh.getNextHouseToVisit();

[ ] House gh2 = hh.getNextHouseToVisit();

[ ] if (gh1 == null) { fail(); }

[ ] if (gh2 == null) { fail(); }

[ ] assertEquals(hh.getNextHouseToVisit(), h1);

[ ] assertEquals(hh.getNextHouseToVisit(), h1);

[ ] assertEquals(hh.getNextHouseToVisit(), null);

[X] assertEquals(hh.getNextHouseToVisit(), h2);

[X] assertEquals(hh.getNextHouseToVisit(), h1);

Code Coverage

Consider this code:

//MODIFIES: this

//EFFECTS: adds the house to the list of houses to visit

// classifying by greatness, dog friendliness, cheapness (over/under $500)

public void maybeAddHouse(House h, boolean great, boolean dogFriendly, int rent){

System.out.println("About to classify house!");

if(great){

housesOfInterest.add(h);

}

System.out.println("I'll settle for allowing my dog");

if(dogFriendly){

housesOfInterest.add(h);

}

System.out.println("what if it's cheap?");

if (rent<500){

housesOfInterest.add(h);

}

}

Statement coverage: (h, true, true, 50)

Branch coverage: (h, true, true, 50) (h, false, false, 600)

How many equivalence classes (individual equivalence classes -- not combinations)? 6: Great, Not Great, Dog, No Dog, Cheap (rent <500), Expensive (rent>500).

Note: These are the individual equivalence classes under the assumption that housesOfInterest is a set. You might also think about the combinations that would be unique. On a real midterm we would either be clear about what we are asking, or accept either answer. You can also watch this video for more context: Equivalence Classes by Reid Holmes

How many boundaries tests needed? 3 to be conservative: pass in rent at 499, 500, 501