Managing Technical Debt

“Do not accustom yourself to consider debt only as an inconvenience; you will find it a calamity” - Samuel Johnson 

A few weeks ago we looked at the problem of technical debt, and at the challenges a Development Team faces when managing it. Technical debt can be defined as the longer term consequences of poor design decisions. We saw that although a Development Team can choose to incur technical debt as a short-term tactical measure, they cannot abdicate responsibility for its ownership. Stakeholders expect a Development Team to do the right thing, and in an agile way of working nobody can force technical debt onto team members. 

The team are wholly accountable for the quality of the work they perform when developing an increment which ought to be fit for release, and they are under no obligation to take on any work which they doubt they can complete satisfactorily. If they choose to take any short-cuts whereby quality is compromised, for whatever reason, then they will be trusted to make the situation good in a timely and responsible manner. 

A technical debt register can provide much-needed transparency over the problem. It can help a Development Team to make informed decisions about whether or not to take technical debt on, to ascertain how much they may have accrued so far, and to plan how and when to pay it back.

We also considered the degree to which technical debt might be articulated upon the Product Backlog. We saw that while this can be an option in certain cases, the Product Owner is under no obligation to accept a Development Team's debt, and may simply not be in a position to do so. By definition the issues are technical in nature, and they may not be understood outside of the team. The incurring of technical debt is a Development Team decision, and it is one in which the Product Owner and his or her stakeholders are not necessarily franchised. If the debt can be understood by the owner, and prioritised in relation to other system requirements, then it may reasonably be managed on the Product Backlog. Any decision concerning repayment will then become a business one. This can be a good thing, since the debt ends up being rationalised in terms of business value. However, parties outside of the team may not be willing or able to help manage technical debt. In such cases, a technical debt register will allow the debt problem to be handled at the appropriate technical level by the Development Team.

Now, it is obviously very important to understand the size and scale of any technical debt which has been incurred. Furthermore, this must be accounted for on the Product Backlog, irrespective of where the debt itself is recorded and described. The Product Backlog must show in its entirety how much work is believed to remain before the product is complete. In other words, the amount of technical debt which has been accrued must be shown on the Product Backlog even if that debt is described elsewhere, such as on a technical debt register. The team should avoid quantifying technical debt on the register itself, as this would lead to a scattered view of the truth.

How though, can the work recorded on a technical debt register be quantified on the Product Backlog? After all, there isn't necessarily a clear relationship between descriptions of technical debt on a register and Product Backlog Items (PBI's). For example, suppose that the team decided to delay some important refactoring in order to facilitate an early release. Technical debt starts to accrue at that point, and it may not map cleanly to specific items on the Product Backlog. It is far more likely to impact components which cross-cut a range of features. Moreover, if the necessary refactoring is significant it could impact the entire product, and it could affect features in uneven ways. What should we do? Things could get messy very quickly if we try to map debt to backlog items and their estimates by means of some formula.

The answer, of course, is that the repayment of technical debt does not necessarily map to specific PBI's at all. What it must certainly map to however is the delivery of increments at least once per Sprint. That's the key. In agile practice technical debt is amortised not in terms of backlog items, but rather in terms of increments which provide actual value. To understand this, remember that in an empirical way of working only incremental deliveries are real, and it is only through the delivery of those increments that can debt be repaid. Also, bear in mind that a backlog is a plan and a forecast of future work. When the Development Team help to refine the Product Backlog, and make their estimates, they should take into account how and when they intend to pay any technical debt off. A good team, like a good housekeeper, will wish to pay off any debt as quickly as possible so it is not compounded further. Hence the team can be expected to amortise their refactoring debt not over the entire backlog in accordance with some bizarre calculation, but only over those items which they forecast for delivery in the very near future, preferably during the next Sprint. That's where and when the effort in refactoring, and in repaying the technical debt, will be invested. If the associated items are reprioritised into different Sprints in subsequent refinement sessions then the estimates ought to be updated accordingly.

Some teams, upon realising that technical debt is amortised by increment, are tempted to commoditise it. This way all hell lies. It might take the form of a quite elementary scam, in which the inability to create a "done" increment is rationalised by norming undone work, and farming its completion out to another team. For example, if organisational gravity and institutional culture forbid a Development Team from doing its own integration or penetration testing, a deficit for release might be not only be recorded but expected. It could then be the responsibility of a so-called "deployment" or "security" team to complete the work. They supposedly "pay off" the debt, which is no longer a team choice or a rare occurrence, but systemic to the organisation's delivery model. In effect, the shenanigans are an implementation of the waterscrumfall antipattern. The important thing to remember is that any complicated model of repaying technical debt ought to be avoided. In this regard, technical debt is like any other debt. Good honest housekeeping is the way to deal with it: keep it simple, keep it exceptional, and keep it under control.