Technical Spikes in DevOps

First published in The DevOps Zone, 12 December 2016

"Two spikes would be an extravagance."

- Lady Whiteadder ("Blackadder," Series II)

It always was the elephant in the room. Now it can no longer be ignored. As if to defy all reasonable laws, including those of common decency and basic physics, the beast has been pushed by management into the executive elevator, trumpeting wildly as it rides up the backlog to the very top. The doors slide apart and it looks out mournfully at the DevOps team, waving its trunk for assistance. An "Urgent!" sticky-note has been slapped rather indecorously onto its hide.

The team groans. For months, this creature was kept down at the bottom of the backlog, deprioritized, and ignored. There was always the hope that it might somehow wander out of the office and back to the business department, back to the circus. No one wanted to deal with it because it brings too many unquantified dangers. What to do now, though, since it cannot be put off any longer? There are great risks and uncertainties to this item, the prospect of unmastered technologies, new skills to learn, and unproven design assumptions being brought into play. No one can say exactly what will be involved, how long it will take, or even if the team can do this kind of work at all.

The lesson, of course, is that these items must be handled carefully from the beginning. As any good agile team knows, that's where a "technical spike" comes in.

In simple terms, a technical spike is a separate refinement activity that allows an ungainly and unquantifiable backlog item to actually be sized. Unlike most backlog refinement work, a spike will typically involve a measure of coding, of "getting some dirt under the fingernails." This is so the team can discover, through hands-on experimentation, what sort of technical work is likely to be required. It allows them to gain insight into the size and shape of the problem. In a spike, any more coding or testing than is needed for estimation is a waste.

Surprisingly, perhaps, a true spike will not contribute any value to the increment that is currently being worked on. For example, if the team is working in sprints, then a spike will not add value to that sprint. Instead, the spike will allow backlog items to be sized for possible planning into future sprints. It is a refinement activity and not a development one. Remember that neither design nor coding nor testing are contraindications to backlog refinement, but they can help to expedite it. Value, on the other hand, will only be added if the team agree to perform the actual development work at a later point. Then it can be planned and actioned to the satisfaction of the Definition of Done and to the actual standard needed for release. A technical spike allows the team to decide how, when, and even if that piece of work will be done at all.

In DevOps and Lean Startup terms, this means that there can be no Minimum Viable Product developed in a spike because nothing is being prepared that will be of release quality. To the extent that a spike may involve prototyping, the prototype will be a "throw-away" one. There can be no hypothesis to test and no actionable metrics based on customer-derived empirical evidence. In a DevOps context, any spike code will not be committed. The ability of a genuinely agile team to throw away spike code, because they know it is of inadequate release quality and serves another purpose, is a sign of their maturity. As always the real value lies in the information gained. Once the actual work is understood it can be prioritized sensibly. It can then be brought into progress at the optimal time, with an appropriate hypothesis formed and a suitable MVP planned for its validation.

When is a spike not a spike? When it tests a hypothesis that is of value to business and can be ordered by them on a backlog in relation to other work. For example, feasibility spikes, which validate a "do-ability" hypothesis, ought to be valued by the customer or Product Owner and prioritized by them accordingly. Wherever possible all work performed by a DevOps team should provide incontrovertible business value. The true technical spike is a last resort and ought to happen only infrequently. It is a step taken when business just wants something done, but the team needs to explore and find out more before that backlog item can be progressed.

In summary, a technical spike is a refinement activity. All refinement ever does is to allow the backlog to be understood, queried, ordered and sized, so it is adequately framed for potential future planning. Any more than that is a waste. The broad objective is to de-risk planning by making it as non-transactional as possible between the DevOps team and their customers. The team should be in a position to say "yes, we can do this" or "no, we can't," and with no need for hedging. There are interesting edge cases such as technical feasibility spikes. These aren't a valid refinement activity because they imply a hypothesis (i.e., "Is this even possible?") and a corresponding MVP. They should go on the Product Backlog because business stakeholders should care about them and the implications of any outcome on value.