Manual Mocking - Resisting the Invasion of Dots and Parenthesis

Colin Goudie about 1 hour later:

• • Isn’t that a bit of a strawman example? I mean, in reality your going to write a lot more tests than 1, that needs that dependency. Then what are your options?

  • 1. With the mockist approach, you’ll be doing one line changes in the tests to setup different expectations or returns. (ideally refactored into nice reusable methods)

  • 2. Manually roll the mocks and end up with weird flag type objects to return different results or worse, lots of hand-rolled mocks of the same interface for different tests

Esko Luontola about 2 hours later:

• • You don’t even need to type

  • Matcher<Object> falseMatcher = new Matcher<Object>() {};

  • It’s enough to type

  • Matcher<Object> falseMatcher = new

  • and then press Ctrl+Shift+Space. IntelliJ will generate the anonymous inner class and stubs for all the methods that need to be implemented.

Esko Luontola about 2 hours later:

• • • I was forced to use a mocking framework because I did not have the source code of the classes I was mocking.

  • In Java it does not matter even if you don’t have the sources for some class. You can anyways extend the class/interface, because you have the .class files for it (otherwise you could not even compile). The lack of sources becomes a problem only when debugging or trying to understand how some third-party code works.

Adam Sroka about 3 hours later:

• • It’s not that the mock approach doesn’t test the behavior of the system. We test the behavior of the object we are currently testing. We test that the current object “dances” nicely by asserting that it obeys the contract of the interfaces it talks to.

  • We do it this way so that we only have to test the behavior of a class once, in one place. It is about DRY. Testing the behavior of our classes in more than once place is:

  • 1) expensive – wasting cycles in our build, which we want to be fast so we can run it a lot.

  • 2) allows the build to lie to us – More than one test failed, but only one behavior is broken. Which one?

  • 3) brittle – when I change something multiple tests might have to change, whereas with the mock approach only two tests must change (The test that uses the mock interface, and the test that exercises the real implementation in the same way.)

  • The only disadvantage is the discipline implied by that last point. If I expect to be able to use an interface some way I had better assure that I have tested using the implementation the exact same way. Even in a strongly typed language I can’t rely on the compiler to force me to do that.

Pablo about 3 hours later:

• • @Esko

  • >> You can anyways extend the class/interface

  • It’s not that easy, if It is a class you have to redefine each method so that the class acts as a mock (a lot of work), and sometimes (e.g. Spring MVC Controllers) the methods are final and you’re screwed.

Steve Py about 4 hours later:

• • I don’t think I understand how you can relate “how mocks are used” with “how mocks are provided”. Personally I use Moq for .Net and I cannot imagine going back to hand-rolling mock objects. Whether people use mocks to see how assemblies dance, or mock around behaviour is irrelevant to how you choose to mock.

  • Hand rolling is ridiculous when dealing with interfaces that have more than a couple methods defined within them, I don’t care how much the IDE fills in for me. That’s like saying “I don’t need Generics/Templates because the IDE/Resharper/CodeSmith can just generate what I need.” It just adds to the cruft that I have to skip over/ignore.

  • As for “dancing”, any checks a Mock does should be as simple and generic as possible. But asserting that key methods are called, and equally important “not” called in a given scenario is important. I.e. if in a normal path dependency A is called to perform an action, and you are asserting behaviour on an alternate path, then the mock can, and should report back a failure if that action is attempted when it shouldn’t have been.

Ben Rady about 5 hours later:

• • Do what is simpler. In Bob’s example, hand-rolled is simpler. Other cases may be different. I don’t understand why this has to be a religious debate.

Adam D. about 5 hours later:

• • Dear Uncle Bob,

  • Take a look at RhinoMocks and AutoMocking container. It’s not Java so your arguments presented here don’t play out the same way in .NET. I find there is more work rolling your own, even with Resharper (what makes Visual Studio act like IntelliJ IDEA).

  • Let me know what you think.

  • Adam

Joon about 7 hours later:

• • Hi Bob

  • I appreciate the “what makes sense” approach of this post.

  • I think that too often we get carried away by shiny things, and forget that there may be less sexy, but more practical ways of achieving tasks.

  • I am a firm believer in doing what makes sense.

  • Having said that, I do like my rhino mocks in .NET ;-)

  • Regards, Joon

Franco Lombardo about 9 hours later:

• • Great post, as usual!

  • A little footnote. I’m and “old style” tester, so I do not appreciate verbosity in tests that does not lead to better expressivity. So what is the advantage of

• assertThat(selection.size(), equalTo(0));

  • compared to

• assertEquals(0, selection.size());

  • ???

Dagfinn Reiersøl about 9 hours later:

• • It seems to me it would be helpful to keep concepts separate. As you point out, Uncle Bob, the example is a stub, not a mock. Mock frameworks exist to generate mocks, not stubs, but people use them for stubs anyway, mostly because they’ve gotten used to it. It’s a capability of mock frameworks that’s used because it’s there. You might argue whether it’s better to use the mock framework or not in specific cases, but the bottom line is the gain is marginal, if it exists at all.

  • It’s different if you need sensing. That means you need a mock or a Test Spy, and those are harder to hand-code.

  • On the other hand, spies are superior to mocks in every conceivable way as far as I can tell. That might give you additional incentive to hand code, since mock frameworks often aren’t capable of generating spies.

Steve Freeman about 11 hours later:

• • C’mon Bob, you can make the case better than this.

  • - as everyone has pointed out, this example is really about stubbing, so can we get the title right? - if I was doing this, I’d probably declare the FalseMatcher as a constant since it has no state. - on the other hand, I’m assuming you don’t care about which object is passed to the Matcher, since you don’t test that. Here it doesn’t matter, in other cases it might. - hand-rolled is only easier to read if you haven’t learned your tools (we’re all craftsmen, right?). How JUnit runs isn’t immediately obvious from the code, that’s why people get confused over when instances are created.

  • As we wrote in the book, testing external libraries is about the last place we’d use a mocking framework, unless we were really stuck.

Steve Freeman about 11 hours later:

• • @Dagfinn Of course I’m biased, but I can’t follow your line about spies and mocks. I find that spies don’t highlight the object relationships as clearly as I’d like, it’s harder to express sequence and state constraints, and they don’t force a failure until after the test is over.

Josh about 12 hours later:

• • When I first started doing TDD (.Net) I hand rolled all my mocks/stubs and that was great because I was just learning the basics of TDD. Introducing a mocking framework at that point would have only frustrated and confused me.

  • That being said, once I stopped to ask myself the question “There has got to be a better way…” I found RhinoMocks. I now use only a speckling of hand rolled mocks, but it is done to make my life easier. Hand rolling is great for specific scenarios where a mocking framework would over complicate stuff, but it tends to be the exception not the rule.

  • It is worth noting however, that there is still a chunk of “pre-RhinoMocks” test code filled with hand rolled mocks that is rarely touched. That at least speaks to what Bob was getting at when he said that our interfaces should rarely change.

Reflective surface about 13 hours later:

• • Anyone who continues to think that using a mocking framework slows you down is living in the stone age. Sorry, that’s just the truth. Using a mocking framework does not slow you down, it speeds you up.

  • Look, using a mocking framework is not my religion, it is one of my disciplines. It’s like dual entry bookkeeping for accountants, or sterile procedure for surgeons. Professionals adopt such disciplines because they understand the theory behind them, and have directly experienced the benefits of using them.

  • I have experienced the tremendous benefit that using a mocking framework has had in my work, and I have observed it in others. I have seen and experienced the way that using a mocking framework helps programmers conceive their designs. I have seen and experienced the way it documents their decisions. I have seen and experienced the decouplings imposed by the tests, and I have seen and experienced the fearlessness with which people using a mocking framework can change and clean their code.

  • To be fair, I don’t think using a mocking framework is always appropriate. There are situations when I break the discipline and handroll mocks. I’ll write about these situations in another blog. However, these situations are few and far between. In general, for me and many others, using a mocking framework is a way to go fast, well, and sure.

  • The upshot of all this is simple. Using a mocking framework is a professional discipline. Using a mocking framework works. Using a mocking framework makes you faster. Using a mocking framework is not going away. And anyone who has not really tried it, and yet claims that it would slow them down, is simply being willfully ignorant. I don’t care if your name is Don Knuth, Jamie Zawinski, Peter Seibel, or Peter Pan. Give it a real try, and then you have the right to comment.

  • Let me put this another way. And now I’m talking directly to those who make the claim that using a mocking framework would slow them down. Are you really such a good programmer that you don’t need to thoroughly mock your work? Can you conceive of a better way to check your work than to express your intent in terms of an executable mock test? And can you think of a better way to ensure that you can write that mock test other than to mock it using a framework?

  • If you can, then I want to hear all about it. But I don’t want to hear that you handrolled a few mocks after the fact. I don’t want to hear that you manually check your code. I don’t want to hear that you do design and therefore don’t need to use a mocking framework. Those are all stone-age concepts. I know. I’ve been there.

  • So there.

Dagfinn Reiersøl about 14 hours later:

• • @Steve Freeman: Thanks, I want opposition to the idea that spies are superior to mocks, since I haven’t really discussed it much. Do you have any examples that would make your position clearer? I admit I haven’t actively sought to compare the two side by side. What seems clear to me is that spies 1) can test anything a mock can test 2) make it easier to test complex objects that are passed to them, since it’s easier to test an actual object than to specify what you expect of a hypothetical object and 3) make for a more natural sequence in the test itself and lend themselves to refactoring to given-when-then.

Stone Age Programmer about 14 hours later:

• • Great comment by Reflective Surface.

  • Nevertheless, this was a balanced article from Uncle Bob. It seems that the experience of the TDD articles has led to this toning down of unsubstantiated claims from Uncle Bob. [Who says an old dog can’t learn new tricks? :-)]

Esko Luontola about 16 hours later:

• Dagfinn Reiersøl said:

  • • Thanks, I want opposition to the idea that spies are superior to mocks, since I haven’t really discussed it much. Do you have any examples that would make your position clearer?

• • One point that I’ve noticed with regards to mocks and spies, is that organizing the tests in a self-documenting way is easier with spies.

  • In a typical mock framework, you set the expectations for a mock in the test setup. When multiple tests use the same setup, then all of those tests fail when one of the expectations fails. So the tests are not isolated (http://agileinaflash.blogspot.com/2009/02/first.html) so that each test would specify only one behaviour, so that each test would have only one reason to fail. Also, the behaviour tested by the expectations is not immediately visible, unless the expectations are extracted to their own method (which is then called from the setup), which in turn makes the test code’s structure messy, and does not make the expectations visible in the test output the same way as all other test methods.

  • Here is an example of how I’ve written the same tests using Mockito (a spy framework) and EasyMock (a mock framework). Make a diff between these two to see the difference:

  • http://github.com/orfjackal/tdd-tetris-tutorial/blob/step7-done/src/test/java/tetris/RemovingFullRowsTest.javahttp://github.com/orfjackal/tdd-tetris-tutorial/blob/step7-done/src/test/java/tetris/easymock/EasyMockExample_RemovingFullRowsTest.java

  • In the EasyMock version there are lots of compromises that had to be made, in order to make the tests read like a specification:

  • - In the first two fixtures, the expectations had to be extracted to their own method and called from the setup, in order to make it clear that what is the behaviour being specified. This way it’s possible to see the intent when reading the test code, but it won’t be visible in the test output when the tests are run.

  • - In the last fixture, the test structure had to be reordered very much. In the other tests the “Arrange” and “Act” are done in the fixture setup, and each “Assert” is done in the fixture’s tests. But in this case the “Act” had to be moved away from the setup. Also an ugly comment was necessary.

  • In short, the use of a mock framework forces a different kind of style of testing on the programmer, so that writing tests in the preferred style is very hard or even impossible. But a spy framework makes it possible to write tests in the same state-based style as all the other tests, while at the same time getting the benefits of interaction-based verification.

Robert Penner about 17 hours later:

• • A mock does verification. You’re not mocking, you’re stubbing.

Lior Friedman about 17 hours later:

• • I can see the benefit of rolling out manual mocks in very simple scenarios. however i do think that in most cases using a mocking framework actually increases the test readability: 1) As Steeve mentioned, anyone having minimal experience with any mocking framework will find the two tests equally understandable. And since mocking frameworks are useful all professional should posses such minimal understanding.

  • 2) using a framework brings together the test and the fakes behavior, I found this improves my understanding of the test. I think it relates to the fact that this tends to make the test linearly readable (no need to jump to and back to the fake actual code)

Kyle Szklenski about 17 hours later:

• • Reflective Surface’s comment was NOT good. It was terrible. No one, not Bob and no one in the comments as far as I can remember (pretty busy right now, so may have missed/forgot one), said anything about using mocking frameworks making you go slower. It was about the expressivity of the test, and how using mocking frameworks often makes them look much uglier, and harder to read.

Bill Sorensen about 18 hours later:

• • It depends. Sometimes hand-rolled stubs are easier, other times an isolation/mocking framework can save significant effort. Especially when you have to test behavior (I’d much rather test state, but sometimes you have to), a mocking framework will verify your assertions for you. I’ve hand-written mocks, and these classes can get out of hand quickly.

  • If your test has too many dots and parens, why not extract the mock creation out into a getFalseMatcher() method? That gives you the best of both worlds.

Josh Brown 1 day later:

• • Along the lines of what Ben said – for simplicity, I like to create my own mocks/stubs when I’m testing Java code, but when I have the option, I like to create them in Groovy. Groovy makes mocking/stubbing so much easier and cleaner than Java, and allows you to use metaClass to pick and choose which methods to replace.

Uncle Bob 1 day later:

• • • Reflective surface about 13 hours later:

    • Anyone who continues to think that using a mocking framework slows you down is living in the stone age.

  • <grin>

msuarz 1 day later:

• • The second test hides the nastiness inside a fake class called FalseMatcher … It is easy to refactor the first test into a method FalseMatcher() or something cool like Actor.FalseMatcher and hide in there the invation of dots and parenthesis. Eg refactor into FalseMatcher()

  • public FalseMatcher falseMatcher() {   Matcher falseMatcher = mock(Matcher.class);   when(falseMatcher.match(anyObject())).thenReturn(false);   return falseMatcher; }

  • @Test public void falseMatcherShouldSelectNoElements_mockist() {   Selector selector = new Selector(falseMatcher());   List selection = selector.select(list);   assertThat(selection.size(), equalTo(0)); }

Joseph Gutierrez 1 day later:

• • When doing .NET I use Rhino. I haven’t seen anybody talk about using dynamic mocks.

  • When TDDing JavaScript, I roll my own because of the use of closures. Functions being first-class definitely makes my life easier and less of a debate of using classicist/mockist.

Philip Schwarz 3 days later:

• • @Reflective Surface Hats off to your choice of name: its irony only struck me as I finished typing this.

  • @Kyle Szklenski You Said: Reflective Surface’s comment was NOT good. It was terrible.

  • If your post is genuine, e.g. you are not the same person as Reflective Surface, or an accomplice of his, then note that his comment was obtained from Uncle Bob’s Echoes from the Stone Age post simply by replacing ‘TDD’ with ‘mocking’ framework.

  • Otherwise… Reflective Surface: I have just fallen victim to your artifice.

J. B. Rainsberger 3 days later:

• • Oh, Bob…. http://bit.ly/2RAMoH

Philip Schwarz 3 days later:

• • In The Art of Unit Testing, which according to Michael Feathers (who wrote the Foreword) is ‘an important title that should have been written years ago’, Roy Osherove, the author, says:

  • Although there are many advantages to using isolation frameworks [a set of programmable APIs that make creating mock and stub objects much easier], there are some possible dangers too. Some examples are overusing an isolation framework when a manual mock object would suffice, making tests unreadable because of overusing mocks in a test, or not separating tests well enough.

  • He also says:

  • Learn how to use the advanced features of an isolation framework… and you can pretty much make sure that anything happens or doesn’t happen in your tests. All you need is for your code to be testable. 

  • You can also shoot yourself in the foot by creating overspecified tests that aren’t readable or will likely break. The art lies in knowing when to use dynamic versus handwritten mocks. My guideline is that, when the code using the isolation framework starts to look ugly, it’s a sign that you may want to simplify things. Use a handwritten mock, or change your test to test a different result that proves your point but is easier to test.

Philip Schwarz 3 days later:

• • @Franco Lombardo

  • Which assertion better communicates that ‘it is true that the selection size is zero’ ?

  • If we use

  • assertEquals(0, selection.size()); 

  • it reads as ‘it is true equals zero selection size’. Not too clear. Partly because ‘equals’ is used as a prefix operator, as in the LISP-like (= a b), and partly because the result operand comes before the computation operand, as in the LISP-like (= result computation).

  • If we use

  • assertThat(selection.size(), equalTo(0)); 

  • it reads as ‘it is true that selection size equal to zero’. That is clearer. The equals is now in infix position, as in a = b, and the computation comes before the expected result: computation = result. But we can do better.

  • If we use

  • assertThat(selection.size(), is(equalTo(0))); 

  • it reads as ‘it is true that selection size is equal to zero’, which is pretty much what we are trying to communicate.

  • Which one do you prefer?

  • As it says in this Hamcrest tutorial, the ‘is()’ matcher is pure syntactic sugar, to make your tests as readable as possible.

Erik Przekop 3 days later:

• • I think the comments point out clearly that this is a “religious” argument to some extent.

  • I use EasyMock very often, and I think I need to write more hand-rolled mocks / stubs. At the very least I need to try it and see how well it works for a given situation.

  • I certainly don’t agree with the assertion that boils down to “good programmers should be able to read code using mocks” (which is what I took away from Reflective Surface’s post). That’s as bad as saying “it was hard to write, so it should be hard to read.”

  • If a mediocre programmer can’t make sense of your code, then it could (and usually should) be written more clearly.

  • I really need to give Mockito a try, though. I’m not very keen on doing behavior-based verification unless I absolutely have to. My (admittedly stone-aged) mind finds it much easier to understand tests that verify state.

Philip Schwarz 4 days later:

• • @Uncle Bob

  • In The Art of Unit Testing, Roy Osherove says:

  • In a test where you test only one thing (which is how I recommend you write tests), there should be no more than one mock object. All other fake objects will act as stubs.Having more than one mock per test usually means you are testing more than one thing, and this can lead to complicated or brittle tests.

  • Your test credentialsShouldBeNonNullIfServiceNamePresent conforms to this practice: it has one mock called manager, and two stubs called gssName and gssCredential.

  • Do you think it would enhance readability to have mock names contain ‘mock’, and stub names containing ‘stub’, like in the Apps flavour of Hungarian Notation?

  • e.g. 1: mockManager, nameStub and credentialStub 

  • e.g. 2: mockManager, stubbedName and stubbedCredential.

  • We would then have:

  • when(mockManager.createName(anyString(), (Oid) anyObject(), (Oid) anyObject())).thenReturn(stubbedGssName); ... when(mockManager.createCredential((GSSName) anyObject(), anyInt(), (Oid) anyObject(), anyInt())).thenReturn(stubbedGssCredential); ... NegotiateAuthenticator authenticator = new NegotiateAuthenticator(mockManager, properties); ... verify(mockManager).createName("service", serviceNameType, mechanism); ... verify(mockManager).createCredential(stubbedGssName, GSSCredential.INDEFINITE_LIFETIME, mechanism, GSSCredential.ACCEPT_ONLY); ... assertEquals(stubbedGssCredential, authenticator.getServerCredentials());

  • This could also help to quickly check that there is no verification of methods called on stubs.