Excuse me sir, What Planet is this?
Excuse me sir, What Planet is this?
Posted by Uncle Bob on Thursday, November 05, 2009
Update 12 hours later.
I’m not very proud of this blog (or as one commenter correctly called it “blart”). It is derisive, sneering, and petulant. It is unprofessional. I guess I was having a bad morning. I slipped. I didn’t check with my green band.
So I apologize to the audience at large, and to Cashto. You should expect better from me.
I thought about pulling the blog down; but I think I’ll leave it up here as an example of how not to write a blog.
Some folks on twitter have been asking me to respond to this blough (don’t bother to read it right now, I’ll give you the capsule summary below. Read it later if you must). It’s a typical screed complete with all the usual complaints, pejoratives, and illogic. Generally I don’t respond to blarts like this because I don’t imagine that any readers take them very seriously. But it appears that this blelch has made the twitter rounds and that I’m going to have to say something.
Here are the writer’s main points:
He likens unit tests to training wheels and says you can’t use them to win theTour de France.
I think winning the Tour de France has much more to do with self-discipline than he imagines it does. I mean it’s not really just as simple as: “Get on a bike and ride like hell!”
He says testing is popular amongst college students
I’d like to see his data!
He goes on to say: (cue John Cleese): “(ahem) unit tests lose their effectiveness around level four of the Dreyfus model of skill acquisition”.
(blank stunned stare). Is this a joke? All false erudition aside, WTF is he talking about?
He says that unit tests don’t give us confidence in refactoring because theyover-specify behavior and are too fine-grained.
He apparently prefers hyphenations to green bars.
He says they mostly follow the “happy path” and therefore don’t find bugs.
Maybe when he writes them! This is a big clue that the author can’t spellTDD.
He complains about Jester
without getting the joke!
He says unit tests “encourage some pretty questionable practices.” He flings a few design principles around and says that unit testing doesn’t help with them.
as the author, editor, and/or advocate of many of those principles; I have a slightly different view.
He says that “many are starting to discover that functional programming teaches far better design principles than unit testing ever will”
Oh no! Not the old “My language teaches design.” claim. We’ve heard it all before. They said it about C++, Java, COM (?!), etc… The lesson of the ‘90s?Languages don’t teach design. You can make a mess in any language.
He says: “tests can have negative ROI. Not only do they cost a lot to write, they’re fragile, they’re always broken, they get in the way of your refactoring, they’re always having you chase down bogus failures, and the only way to get anything done is to ignore them.”
In one of my favorite episodes of Dr. Who, Tom Baker exits the Tardis, walks up to someone on the street and says: “Excuse me sir, what planet is this?”
He says: “What I’m saying is that it’s okay if you don’t write unit tests for everything. You probably have already suspected this for a long time, but now you know. I don’t want you to feel guilty about it any more.”
Translation: “I don’t want to feel guilty about it anymore so I’m going to try to convince you…” I sincerely doubt this author has your best interests at heart.
He says: “Debugging is easy, at least in comparison to writing all those tedious tests.”
Refer back to the Dr. Who quote.
To quote Barack Obama: “Enough!”
Has this guy ever done TDD? I rather doubt it. Or if he did, he was so inept at it that his tests were “fragile”, “always broken”, and “in the way of refactoring”. I think he should give it another try and this time spend a bit more time on test design.
Perhaps he’s one of those guys who thought that unit tests were best written afterthe code. Certainly his list of complains makes a lot of sense in that light. Hint: If you want to fail at unit testing, write them last.
The bottom line is that the guy probably had a bad experience writing unit tests. He’s tired of writing them and wants to write fewer of them. He’d rather debug. He thinks he can refactor without tests (which is definitively false). He thinks he can go faster by writing fewer tests. Fine, that’s his choice. And he’s found a rationalization to support his desires. Great.
I predict that his results will not compare well with those who adopt the discipline of TDD. I predict that after a few years he’ll either change his mind, or go into management.
Oh, and to the author: Gesundheit!
Comments
Mark Nijhof 33 minutes later:
He would make a great manager, one that can tell the developers not to do TDD because he knows they are useless, because he was a developer once and a good one of course who has even written a blog post about how useless TDD is.
When he promotes I would start looking for a different job.
-Mark
kewlito 35 minutes later:
My (short) experience with unit testing says that when a refactoring breaks some tests, it’s because those tests were checking implementation rather than behavior, or because a change in behavior is what we’re looking for (which means that, by Fowler’s definition, you’re actually doing something that isn’t a refactoring). So, if you break some test while refactoring, maybe your tests are not well designed.
Erlis about 1 hour later:
@kewlito
...or your tests are telling you that your refactoring was wrong!
Daniel Sobral about 1 hour later:
I’d much rather you pondered the discussion athttp://www.artima.com/forums/flat.jsp?forum=106&;thread=272118.
A common complain is that refactoring breaks too many tests, resulting in too many costs for the benefit. That would be a nice topic for a post. And, as opposed to this one, useful! :-)
Eduardo Scoz about 1 hour later:
Yes, tests break every once in a while when you’re doing refactoring. You know what?
But you know what? I EXPECT my tests to break/fail when I change a lot of stuff mercilessly.. That tells me that I’m probably touching something I just don’t know about, and reminds me of old bugs that were fixed, old business rules written three years ago, and decisions that were made a long time ago. Knowing that those tests would break is what allows me to refactor a lot of stuff in the first place. I don’t know how people refactor code without a suite of tests in place.
Each test that breaks because of changes made is basically a bug that was never introduced in the system by the changes in the first place. Much better than end up throwing a bunch of crap to the testers.
In my experience, most people who complain about tests are simply people with zero experience writing large enterprise applications, written by dozens of developers during many years.
Elroy about 1 hour later:
MSDN blogs blurting out crap??
I had been eagerly waiting for your response to that crap, but somehow I felt that you’d take him just as a non-follower and let him go. But we like to see Uncle Bob spank the naughty kid.
And, yes the post is quite demeaning to the devoted followers of UT and its benefits. I’m usually surrounded by people who think the same. It’s a lost cause to explain things to them.
I usually ignore these controversial posts. They’re not for me. I follow what I know is best for me and my code and Uncle Bob does an awesome job of reminding us exactly that, time and again.
Cedric about 2 hours later:
Rob,
The article you quote is more about emphasizing the importance of functional tests over unit tests than it is about bashing testing, so I think you’re overreacting a bit here.
He’s mistaken about completely disregarding unit tests, but I sympathize with his point about emphasizing functional tests, which I think should never get neglected because they impact your users (as opposed to unit tests, which impact the developer, and are therefore more ‘selfish’).
I still think that your dogmatism toward TDD is troublesome and clouds your overall message (the quoted article only mentions TDD once, by the way). I wish you spent more time convincing the world that it needs testing (“any testing”) and then let everyone figure out the best way to do that. If it’s TDD, then it will happen naturally.
— Cedric
Cookiecat about 2 hours later:
I, for one, appreciate it when there is some discussion about the costs/benefits of TDD. I personally believe in the merits of TDD if done correctly, but have a hard time convincing others as a result of http://martinfowler.com/bliki/CannotMeasureProductivity.html. Even the TDD case studies done at MS and IBM can be disputed on the grounds that they are: different projects/technologies/experience/team size/code size.
Uncle Bob, rather than condemning some of the assertions, can you provide more explanation? For example, you say
“as the author, editor, and/or advocate of many of those principles; I have a slightly different view.” - What is your view on the effects TDD has on those principles?
“He says they mostly follow the “happy path” and therefore don’t find bugs.” - I understand TDD is intended to prevent bugs, not find them, but can anyone provide more reading on how to avoid this very common pitfall of writing the same mistakes into both code and test?
Jake about 2 hours later:
@Cedric,
Well said. Couldn’t agree more.
Stan about 2 hours later:
You know what cashto’s real problem is … “working on various features now and then, but mostly just a lot of bugfixing.” He likes wasting time debugging and bug fixing rather than trying anything else, maybe he is even like “hey you will put me out of my job” :)
Freddy about 3 hours later:
@Cedric, @Jake and I couldn’t disagree more … that article clearly presents arguments regarding his view that unit tests has all those issues.
Not saying anything about it, is giving into all the wrong statements that are in that post. It has so much of that, that it clouds whatever he was trying to say and makes the whole thing pointless.
You just can’t ignore that. Of course, I wouldn’t use the same tone in uncle bob’s reply, specially because its not that hard for someone not to realize the mistakes they are making when trying to do unit testing and there is no one around to coach. That person should be pointed to clear examples / readings … maybe he will write a post: “what I was missing about using unit tests during development”. developing with unit
Max about 3 hours later:
cashto is not saying that tests are a bad idea, or even that unit testing is. He’s simply pointing out that over-reliance on unit testing is a bad idea. (He doesn’t make his point very well though, I admit.)
A few years back, I designed, implemented and supported a large DB framework handling thousands of transactions per seconds on a 24×7 life-critical system. In five years, I was not called even once for a problem at a customer site. Not once in five years.
How many unit tests did my framework have? A grand total of zero. z-e-r-o.
However, we had a truck load of acceptance tests, load tests, etc. These higher-level tests were not only able to catch unit-test level bugs easily but also systemic bugs, memory consumption bugs, performance bugs, wrong libraries bugs, wrong OS patchlevel bugs…
People must come to the realization that unit testing is not the end all and be all of testing. The TDD religion is wearisome.
Max about 3 hours later:
cashto is not saying that tests are a bad idea, or even that unit testing is. He’s simply pointing out that over-reliance on unit testing is a bad idea. (He doesn’t make his point very well though, I admit.)
A few years back, I designed, implemented and supported a large DB framework handling thousands of transactions per seconds on a 24×7 life-critical system. In five years, I was not called even once for a problem at a customer site. Not once in five years.
How many unit tests did my framework have? A grand total of zero. z-e-r-o.
However, we had a truck load of acceptance tests, load tests, etc. These higher-level tests were not only able to catch unit-test level bugs easily but also systemic bugs, memory consumption bugs, performance bugs, wrong libraries bugs, wrong OS patchlevel bugs…
People must come to the realization that unit testing is not the end all and be all of testing. The TDD religion is wearisome.
William Pietri about 4 hours later:
It’s funny, Max; when I first started reading Uncle Bob’s opinions on testing, the common reaction was that automated testing was never really worth it, or even if theoretically worth it, impossible in practice. If we’re down to arguing about which sort of testing is best to achieve zero defects, then the industry has come a long way, and I think most of that’s due to the Agile movement.
You might also note that Bob is a major force behind Fitnesse, an acceptance test framework. So I’m pretty sure he’s not suggesting that unit testing is the end-all of anything.
What your critique lacks is an indication that you’ve actually tried TDD in a significant way. There are an awful lot of New Yorkers who will tell you that New York is the best city in the world, but if they’ve never lived anywhere else, it’s hard to take their assertion of relative superiority seriously, yes?
Joe about 4 hours later:
This answer is the same blart as the one you are answering…
ErikFranzK about 4 hours later:
@Max: the anti-TDD “religion” is just as wearisome as the TDD-”religion”. “Religious” arguments only let those who use them look silly, whatever side they’re on.
As far as I know there were scientists who never accepted the theory of relativity or quantum mechanic – they died out, mostly. Not to say that TDD is right or ground breaking – only that time will tell what survives…
Mike Wilson about 5 hours later:
Yeah, well… TDD is hard. It’s very pay up front. Frankly it’s hard for a long time and it forces you to give up the smug notion that “you just think better in code.” It’s like people who say they drive better with a couple drinks in them to loosen them up.
No you don’t.
Once you get good at it you begin to have those Michael Feathers moments of “Wow, when would I possibly have found that bug without the tests.”
peekay101@hotmail.com about 6 hours later:
Tom Baker is my favorite Doctor. He would have been a big advocate for unit tests.
Adam Sroka about 7 hours later:
Unfortunately, every one of cashto’s points are valid at some point in the maturity of the programmer writing the tests. Most of us encountered the problems he mentioned.
The difference is that some of us invested the time to solve those problems and get good at unit testing. Cashto decided that the problems outweighed any perceived advantages of unit testing and gave up. He isn’t alone.
As a coach, cashto’s opinions are very interesting to me. He clearly tried TDD and did it very poorly. I want to make sure that the teams I coach try it too, and I want to make sure that they have more help than cashto did.
Steve Py about 9 hours later:
I think the trouble is that his opening statement in response to Michael Feathers’ writing is bloody confusing:
“If they are any dependencies, they are mocked away. Do your tests do that?
That’s what I thought. They’re not unit tests, they’re just tests.”
What is the point he’s trying to get across? That if you’re tests “do” mock away dependencies, they “aren’t” unit tests, or is his assumption that most people “don’t” mock away dependencies therefor they aren’t writing unit tests? Assumptions based on “you think like me, right?” are a really stupid way to try and make an argument for anything.
His assumptions about the purpose of unit tests, whether “pure”, functional, or behavioural are completely and utterly wrong. Unit tests don’t find bugs, they prevent them. Tests are only as good as the people that write them. If you’re breaking tests “unnecessarily” then you need to learn to write better tests and better code. (Adopting IoC etc. to manage dependencies and test in isolation.)
As for his comments about the cost of testing. BALDERDASH. I have this discussion all the time. Developers will say “I have to factor in extra time to write unit tests.” I tell them that they actually don’t. When you give an estimate for a unit of work you surely factor in time to fire up the application and make sure the functionality works as you expect and meets the requirement, non? That is the time you should be spending writing tests to assert the desired behaviour. Writing it as a repeatable test is far, far better than doing that step manually because how often when implementing functionality do you go through the application thoroughly enough to assert that not only this requirement is met, but all previous requirements are alsostill met? If you write it to be repeated, you SAVE a hell of a lot of time that you SHOULD be spending to repeat it. Choosing simply not to repeat it is irresponsible and causes expensive and embarassing problems down the road. IMO if anyone can’t see that, they should switch careers.
Maybe you can argue that these aren’t unit tests by the letter of some law that was written. Follow the spirit, and leave the letter of the law to those that feed on dogma.
Charles about 14 hours later:
Uncle Bob, I think you are awesome and your insights are moving our art forward, but I get really sick and tired of seeing you write nonsense like this:
“The bottom line is that the guy probably had a bad experience writing unit tests.”
You’ve admitted indirection in your “Clean Code” book and directly in your blog that TDD requires a certain moment of “getting it”. I’ve been a professional programmer for 20 years and I do not “get” TDD.
Yet, I understand why you need something like TDD - you need engineering discipline and the confidence to know whether your system works. Well, there are other techniques for this. Ironically, your product FitNesse is one of the best behavioral and adversarial-testing suites around.
Rather than “red-green-refactor”, I tend to test at the “let’s understand how the system works as a whole” level and then go “red-hack-hack-hack-hack-refactor-hack-hack some more-no, that’s not it-start over-hack-hack-hack-hack-green”. By the time I get to green, my code needs little refactoring. But, you know what, even if my code is complete garbage internally, by then I have quite the little suite of behavior-driven tests and can rewrite the system as many times as I want with great confidence, because when the bar is green I know I can freaking ship.
Can you say the same for your unit tests? Well, sometimes you can and sometimes you can’t. But if I express every high-level story and every bug as an entire army of behavior tests, I know that when my bar is green I can freaking ship.
What I’m saying is that while I recognize TDD works for you, I recognize that BDD + CIR + incessant hacking works for me.
I view the Agile movement as a collection of “hey, this might work” practices rather than “do it EXACTLY THIS WAY or OMG you are an idiot”. I think you would do even more for the art if you occasionally thought of it in those terms, too.
Charles about 14 hours later:
“You’ve admitted indirection” is supposed to be “you’ve admitted indirectly.” Sorry about that.
kewlito about 20 hours later:
I found this response interesting:http://www.codethinked.com/post/2009/11/05/Ite28099s-Okay-To-Write-Unit-Tests.aspx
Geir 1 day later:
He says: “tests can have negative ROI. Not only do they cost a lot to write, they’re fragile, they’re always broken, they get in the way of your refactoring, they’re always having you chase down bogus failures, and the only way to get anything done is to ignore them.” * In one of my favorite episodes of Dr. Who, Tom Baker exits the Tardis, walks up to someone on the street and says: “Excuse me sir, what planet is this?”
He is actually right. I have seen lots (ooh, lots) of people lose their way writing unit tests. In extreme cases, they go off and write “testConstructor”-type test, because someone told them they needed a high test coverage, and then Good Stuff would happen. They become puzzled when they can’t refactor their code without significant test breakages.
Often, there is a project manager around who is actively dissuading developers from any quality-enhancing efforts by repeating variations of a “more speed” mantra. The frequency of repetition of the mantra goes up with the amount of delay caused by frantic bugfixing.
I think developers are a lazy bunch. Sometimes, we read the preface to a book, become immediate experts, and then reject the contents of the book based on not getting it to work as advertised. I hope I am not like that too often.
Bill Sorensen 4 days later:
I agree with Adam Sroka’s comment. To me the one overly harsh note in Uncle Bob’s post was “he was so inept at it that his tests were…”
Writing good unit tests is a skill. It is hard. I’ve been doing it for years (with and without TDD) and still feel I have much to learn. I’ve made many of the mistakes mentioned and experienced the associated pain.
So why do I keep doing it? Because I see the benefits. Improved code quality. Improved design. Less time spent in the debugger. Confidence when refactoring. Fast feedback.
Simon Fox 5 days later:
Kind of ironic that reading the About page of the author of the said “blough” you find the following statement:
”...working on various features now and then, but mostly just a lot of bugfixing…”
gilz@typemock.com 6 days later:
Roy Osherove and myself discuss this at our weekly “This week in testing” video cast:
http://learn.typemock.com/this-week-in-test/2009/11/11/episode-3-magic-numbers.html
Kos 6 days later:
At this point, I feel obliged that I have personally put your blog in the category of “blogs of programmers more experienced than me, who I can learn stuff from”.
(please don’t confuse me for Bob Koss – i’m just a random reader)