We must ship now and deal with consequences
We must ship now and deal with consequences
Posted by Uncle Bob on Thursday, October 15, 2009
Martin Fowler has written a good blog about technical debt. He suggests that there are two axes of debt: deliberate and prudent. This creates four quadrants:deliberate-prudent, deliberate-imprudent, inadvertent-prudent, and inadvertent-imprudent. I agree with just about everything in his blog except for one particular caption…
Inadvertent-Imprudent Debt.
There is more of this debt than any other kind. It is all too common that software developers create a mess and don’t know they are doing it. They have not developed a nose that identifies code smells. They don’t know design principles, or design patterns. They think that the reek of rotten code is normal, and don’t even identify it as smelling bad. They think that their slow pace through the thick morass of tangled code is the norm, and have no idea they could move faster. These people destroy projects and bring whole companies to their knees. Their name is Doom.
Deliberate-Imprudent Debt.
There is a meme in our industry (call it the DI meme) that tells young software developers that rushing to the finish line at all costs is the right thing to do. This is far worse than the ignorance of the first group because these folks willfully create debt without counting the cost. Worse, this meme is contagious. People who are infected with it tend to infect others, causing an epidemic of deliberately imprudent debtors (sound familiar?) The end result, as we are now all know, is economic catastrophe, inflation (of estimates) and crushing interest (maintenance) payments. They have become death, the destroyer of worlds.
Inadvertent-Prudent Debt.
This is something of an oxymoron. Ironically, it is also the best of all possible states. The fact is that no matter how careful we are, there is always a better solution that we will stumble upon later. How many times have you finished a system only to realize that if you wrote it again, you’d do it very differently, and much better?
The result is that we are always creating a debt, because our hindsight will always show us a better option after it is too late. So even the best outcome still leaves us owing. (Mother Earth will eventually collect that debt!)
Deliberate-Prudent Debt.
This is the quadrant that I have the biggest problem with. And it is this quadrant in which Martin uses the caption I don’t like. The Caption is: “We must ship now and deal with consequences.”
Does this happen? Yes. Should it happen? Rarely, yes. But it damned well better not happen very often, and it damned well better not happen out of some misplaced urge to get done without counting the cost.
The problem I have with this quadrant (DP) is that people who are really in quadrant DI think they are in DP, and use words such as those that appear in the caption as an excuse to rack up a huge imprudent debt.
The real issue is the definition of the word: Imprudent.
So let me ask you a question. How prudent is debt? There is a very simple formula for determining whether debt is prudent or imprudent. You can use this formula in real life, in business, and in programming. The formula is: Does the debt increase your net worth, and can you make the payments?
People often focus on the first criterion, without properly considering the second. Buying a house is almost certain to increase your net worth despite the debt (though lately…). On the other hand, if you cannot make the payments, you won’t keep that house for long. The reason for our current economic woes has a lot to do with people trying to increase their net worth despite the fact that they couldn’t afford the payments. (indeed, they were encouraged by a meme very similar to the DI meme!)
Bad code is always imprudent.
Writing bad code never increases your net worth; and the interest rate is really high. People who write bad code are like those twenty-somethings who max out all their credit cards. Every transaction decreases net worth, and has horrendous consequences for cash flow. In the end, the vast bulk of your effort goes to paying the interest (the inevitable slow down of the team as they push the messes around). Paying down the principle becomes infeasible. (Just the way credit card companies like it.)
Some Suboptimal Design Decision are Prudent Debt.
But most are not. Every once in awhile there is a suboptimal design decision that will increase the net worth of the project by getting that project into customer’s hand’s early.
This is not the same as delivering software that is under-featured. It is often prudent to increase the net worth of a project by giving customers early access to a system without a full and rich feature set. This is not debt. This is more like a savings account that earns interest.
Indeed, this is one reason that most technical debt is imprudent. If you are truly concerned about getting to market early, it is almost always better to do it withfewer features, than with suboptimal design. Missing features are a promise that can be kept. Paying back suboptimal designs creates interest payments that often submerge any attempts at payback and can slow the team to the breaking point.
But there are some cases where a sub-optimal design can increase your net worth by allowing you to deliver early. However, the interest rate needs to be very low, and the principle payments need to be affordable, and big enough to pay back the debt in short order.
What does a low interest rate mean? It means that the sub-optimal design does not infiltrate every part of your system. It means that you can put the sub-optimal design off in a corner where it doesn’t impact your daily development life.
For example, I recently implemented a feature in FitNesse using HTML Frames. This is sub-optimal. On the other hand, the feature is constrained to one small part of the system, and it simply doesn’t impact any other part of the system. It does not impede my progress. There is no mess for me to move around. The interest rate is almost zero! (nice deal if you can get it!)
Implementing that feature with ajax is a much larger project. I would have had to invest a great deal of time and effort, and would have had to restructure massive amounts of the internal code. So the choice was a good one.
Better yet, the customer experience has pretty much been a big yawn. I thought people would really like the feature and would drive me to expand upon it. Instead, the customer base has virtually ignored it.
So my solution will be to pay back this debt by eliminating the feature. It was a cheap experiment, that resulted in my not having to spend a lot of time and effort on a new architecture! Net worth indeed!
But it might have gone the other way. My customers may have said: “Wow, Great! We want more!” At that point it would have been terrible to expand on the HTMLFrames! That decision would have been in the DI quadrant. Deliberate imprudence! Rather, my strategy would have been to replace the suboptimal Frames design of the feature with an isolated ajax implementation, and then to gradually migrate the ajax solution throughout the project. That would have been annoying, but loan payments always are.
Summary
So, don’t let the caption in the DP quadrant be an excuse. Don’t fall for the DI meme that says “We just gotta bite the bullet”. Tread very carefully when you enter the DP quadrant. Look around at all your options, because it’s easy to thinkyou are in the DP quadrant when you are really in the DI quadrant.
Remember: Murphy shall send you strong delusion, that you should believe you are in DP; so that you will be damned in DI.
Comments
Aaron about 1 hour later:
Question: Suppose a DP decision has been made, such as your FitNesse HTML frames, at what point do you pay the interest? Is it preferred to address the design issue immediately after the release so the next release does not suffer from the issue, or is it better to wait until the next story to address fixing the issue?
It seems in your example that being proactive would have been a loss, especially since there has not been demand to fix the issue. Leaving poorly designed code in the system rubs the wrong way, but seems like better way. Additionally by waiting the correct design may appear from another area of development which would be reusable to correct the design issue.
Stefano Ricciardi about 1 hour later:
Great articles (both yours and Martin’s). A lot of food for thought.
Uncle Bob about 2 hours later:
Aaron about 1 hour later:
Question: Suppose a DP decision has been made, such as your FitNesse HTML frames, at what point do you pay the interest?
The good thing about the Frames solution was that there is almost no interest to pay. The code for this feature is way off to the side, and would be trivial to remove. So the interest cost is virtually zero.
When interest is higher (i.e. you can feel the impediment of the suboptimal design every day that you write code) then it is really important to reduce either the interest or the principle as rapidly as you can. You reduce the interest by decoupling from the suboptimal design. You pay down the interest by improving the suboptimal design.
Daniel about 4 hours later:
Good points…
So whats the next level of technical debt…Technical Foreclosure? Is there such and animal? How would you define that?
Thanks for all the great posts.
msuarz about 6 hours later:
if the ignored feature was the Tree on the left i do not like it … maybe making it collapsible would allow me to use it if i need it … but having it there all the time is a waste of space from my perspective
cheers mike
R Hayes about 9 hours later:
Deliberate Prudent debt is not nearly so rare as you think. If you are chasing a large opportunity with a short half-life, it is imprudent not to accept this debt. If you’re not ready to make hay when the sun is shining, you’re not going to make hay at all.
An excellent example of deliberate technical debt in the real world is Formula One racing. Those cars are a combination of the world’s best technology … and duct tape. If the car has a problem that causes a part to wear out every third race, who cares? The engine gets overhauled more often than that. The car has to finish the race 95% of the time. More than that is probably costing you races.
An example in the programming world is statistical arbitrage algorithms. The framework they sit in may be perfectly engineered, but the implementations of the algorithms rarely are. That additional time is money lost. Worse yet, most of the opportunities they chase evaporate as soon as other people start imitating the strategy.
Keith Nicholas about 10 hours later:
“Writing bad code never increases your net worth;”
this is wrong.
I’ve seen people launch businesses based on their bad code! through the will to get something working they make their code do what they want.
I think the relationship between code quality and net worth is pretty complex.
for instance, anyone who has worked with a mathematician or scientist who have crafted a piece of code to solve a hard problem, often its a mess, but it solves the problem. Money in the bank. It doesn’t mean you can’t improve it, and that there aren’t consequences to having messy / bad code.
You can certainly make the statement for a number of common cases that many people are involved with, but “never” is too strong.
Jaap Beetstra about 10 hours later:
Daniel: I would say Technical Foreclosure is when you lose to the competition due to inability to deliver quality features on-time.
R Hayes: I don’t think F1 racing is an example of technical debt. Part of the specification is: A part must last 1 race or more. If a part wears out every third race, that is acceptable. If the part fails in 33% of the races, it is completely unacceptable. Failure must be predictable in F1.
abby, the hacker chick blog about 11 hours later:
I’m downright teary eyed.
sniff
skyivben about 13 hours later:
Agile Principles, Patterns, and Practices in C# Chapter 6: A Programming Episode
Game.cs refactoring:
// Game.cs —-—-—-—-—-—-—-—-—-—-—-——
public class Game { Scorer scorer = new Scorer(); }
public int Score { get { return ScoreForFrame(10); } }
public void Add(int pins) { scorer.AddThrow(pins); }
public int ScoreForFrame(int theFrame) { return scorer.ScoreForFrame(theFrame); }
MikeT about 14 hours later:
Also, the left hand tree at fitnesse.org doesn’t show up in Google Chrome.
Jon B about 14 hours later:
You can certainly make the statement for a number of common cases that many people are involved with, but “never” is too strong.
To me there are two reasons why its worth saying never even if never is too strong.
The problem with not saying never is that it’s virtually impossible to distinguish between bad code thats benign and bad code thats eating away your net worth until the problem has gotten out of control.
The other reason is that when something inevitably does go wrong there’s a greater space where there could be a problem. If i have lingering doubts about part of my code i’m going to have to spend some time sifting through the chaff to make a decision on whether its the bad code going malignant or if it’s something external. The bad code acts as a fog, making the problem harder to find simply by its presence.
There was a statement I got from an agile diary that was posted on Ron Jeffries’ XP Mag. “To me NO quality means that I cannot guarantee a working application”. If you start accepting bad code you’re not moving down a couple of steps on the quality ladder, you’re saying “I no longer have control” and letting go of the wheel.
So while you may believe ‘Never’ is too strong, it’s more valuable to cultivate an instinct of never than attempting to identify which cases are the exceptions. The smallest benefit of which is that the more experienced at clean code you get the easier it becomes.
Uncle Bob about 18 hours later:
R Hayes about 9 hours later:
Deliberate Prudent debt is not nearly so rare as you think.
I know it’s not rare. I wish it was rarer.
If you are chasing a large opportunity with a short half-life, it is imprudent not to accept this debt.
That’s a false dichotomy. It suggests that debt is the only way to go fast. Generally, technical debt is a good recipe for going slow, even in the short term. There are exceptions; but you have to be very careful about them.
If you’re not ready to make hay when the sun is shining, you’re not going to make hay at all.
By all means make hay while the Sun shines. But cut that hay to nice even lengths, bale it up nice and tight and square, and stack those bales evenly. Otherwise it’ll all fall apart before you get it to market. You can’t make hay by making a mess.
Uncle Bob about 19 hours later:
Keith Nicholas about 10 hours later:
“Writing bad code never increases your net worth;”
this is wrong.
I’ve seen people launch businesses based on their bad code! through the will to get something working they make their code do what they want.
I’ve seen that too. I’ve also seen some of those businesses stagger, crash, and burn because they could not manage the founder’s code.
Again, it is a false dichotomy to suggest that it is faster to write messy code. In fact it’s faster to write clean code.
for instance, anyone who has worked with a mathematician or scientist who have crafted a piece of code to solve a hard problem, often its a mess, but it solves the problem. Money in the bank. It doesn’t mean you can’t improve it, and that there aren’t consequences to having messy / bad code.
There is a difference between code written carefully by an inexperienced programmer (e.g. your scientists), and code written carelessly by someone in a rush. The first may have some deficits in terms of coding style; but it is not likely to become the seed of a disaster. The second, if pervasive enough, will cause unpredictable crashes, and inadvertent breakage, that will slow the development asymptotically to zero.
You can certainly make the statement for a number of common cases that many people are involved with, but “never” is too strong.
I disagree. Bad code, code written carelessly, never increases you net worth.
Torbjörn Kalin about 20 hours later:
Uncle Bob: I wouldn’t call your Frames solution technical debt. It’s just a different way to solve the problem. After all, ten years ago frames was as cool as you could get. Just because a cooler technology has now entered the scene doesn’t make the old technology become a debt.
Technical debt for me doesn’t have to do with what technology you use. It has to do with how you use it.
Also, I would say the difference between reckless and prudent deliberate debt is that, if you have a plan to deal with the debt it’s prudent, otherwise it’s reckless. That is, it’s (sort of) ok to create a mess before the Friday deadline if you clean it up on Monday morning.
Apart from that, a very good post!
Jon B about 20 hours later:
After thinking about the original Martin Fowler article, I get the feeling that Martin would completely agree with you Uncle Bob.
Specifically “A mess is a reckless debt”, no prevarication or wiggle room there. If there’s a code smell then it’s a reckless debt. The only thing left to talk about is was it unintentional (in which case, it’s a learning experience) or deliberate (i’ll leave the consequences as an exercise to the reader).
To me the prudent debt axis talks more about sub-optimal design decisions.
To borrow an example from “Agile Software Development” (p103-108), We’ve got good clean code to draw shapes regardless of type. While waiting near the water cooler for the build server to give the code the all clear, coder 1 turns to his pair and says “You know we’d probably be better off with a data-driven table based approach. We know we’ve got that ‘draw all circles before squares’ story later in the pipeline. why don’t we just implement that now.” to which coder 2 says “YAGNI. We’ll do that when we come to it”.
Is this a debt? I’d certainly argue that it is, even if it’s probably not valuable to present to the customer in that way. We’re borrowing against future work to maintain the simplicity of the system (or alternatively to defer a decision until the last responsible moment). This is what i’d categorise as a deliberate/prudent debt. Deliberately accepting a less than optimal but still viable(i.e. no code smells) system design.
The interesting question to me is then “Is it better to maintain this kind of debt or spend the time to eradicate it immediately”?
Luca Minudel 5 days later:
> Bad code is always imprudent.
When we as developers write bad code, is not because it is faster ann we are late. This is just a lie we tell to ourself. In reality it is because we are not able to wrirte better code!
Writing good code do not require more time, it just require more skills.
That’s the point. Plain simple.
Marko 5 days later:
Fully agree with Luca.
Ronie Uliana 11 days later:
Luca +1.
And 10 years is barely enough to get the right skills (emphasis on “right”, it’s easy to focus on the wrong ones)