Method Wars: On the Commoditization of Estimates

First published in The Agile Zone, 9 September 2013

In theory there is no difference between theory and practice, but in practice there is.- Yogi Berra

Some background

A couple of weeks ago, Dean Leffingwell broke his silence on Ken Schwaber’s recent blast about the Scaled Agile Framework. For those of you who haven’t been following this latest skirmish in the method wars, Ken – the co-founder of Scrum - tore into SAFe and dismissed Dean’s creation as a dangerous and essentially unagile spin on RUP. There’s politics involved in all of this of course, but there’s also a technical matter at its heart. Scrum is a conceptually simple framework, very clean, and remarkably non-prescriptive in the advice that it gives regarding its implementation. Yet as a good model, it should in theory scale well, and remain entirely applicable in multi-team environments at an enterprise level. SAFe has a different philosophy behind it: the real world is a messy place, and we cannot be naïve about this; we must extend and tailor model-theoretic approaches to suit the complex and often dirty conditions we have to deal with in practice.

I know I’m generalizing, but that’s essentially the context behind this dispute. Can Scrum theory, as far as it goes, really scale in a practical way? SAFe advocates have argued otherwise. On the other hand, is the theory behind SAFe too compromised to be viable as a practical agile framework? Some hard-core Scrum advocates are inclined to think so.

This might seem an awkward place to draw battle-lines, because we would expect agile theory and practice to meet. Yet as Yogi Berra rightly pointed out, in theory there is no difference between theory and practice, but in practice there is.

Into the Fray: The Normalization of Estimates

I’ve already documented my own thoughts about SAFe and some of the issues I found in applying it. One of the experiences I touched on was the thorny matter of comparing story points across teams:

Normalizing story points between teams robbed them of control of their own process. Their metrics were no longer their own for the purpose of inspection and adaptation. Instead they were used by management, ostensibly to facilitate planning, but also with a view to comparing teams. There was a real danger of "point-productivity" becoming more important than the actual delivery of potentially releasable increments. SAFe's contention that "with adjustments for economics of location, (US, Europe, India, China, etc.), work can be estimated and prioritized based on economics by converting story points to cost" is hard to consider with a straight face.

Dean was gracious enough to reply:

Velocities are NOT normalized across teams. Estimating is. If you don’t normalize estimating, then there is no meaningful economics; you can’t figure ROI if you don’t know that the “I” is. If you want to scale agile, and there is no meaningful way to bid work across teams, and within a program, you will be blocked before you even try. (Executive view: “That agile thing; not ready for prime time. They use gummy bears per sprint as their main metric, and they can’t estimate work in any meaningful way. They just don’t get it. I know, let’s just have the PMs do a work break down structure; that we understand. And of course, we’ll need good specs to do that, so let’s stop and write the SRS now”.) Then Agile loses.

Here we get to the nub of the issue. Dean is accepting the fact that managers have certain prejudices, and that Agile will “lose” if we don’t make certain compromises in order to accommodate them. It’s arguably a very pragmatic position to take. Scrum, on the other hand, takes a stance that is more in line with essential agile principles. You can only shift so far, then you are not being agile at all. Normalizing estimates is unacceptable, because in an agile world the value does not lie in, and should never be mistaken for, any forecasts that might be made. Agile practice is founded in empiricism, where the proof of success can only come from delivery.

Here’s what Johanna Rothman told me about using – or rather misusing – estimates for this purpose:

Placating management doesn’t change the conversation. We have to help management understand why value is part of the conversation, along with incremental funding. And that means technical teams have to do their part, and deliver value on a regular basis.

The Commoditization of Estimates

What Ken Schwaber said to me indicated that he was on the same page.

The only reason for estimating is for a team to get its arms around how much they should try to take on.

Of course he’s right, in so far as that represents a canonical take on what estimates should be used for in an agile way of working. Where Ken astounded me was in making the following very perceptive remark.

I’ve found that estimates cannot be commoditized.

That’s a wonderful interpretation, because it describes what is happening so succinctly. In SAFe, estimates are treated as being a tradable product of sorts, in so far as work can be bid across teams. That’s commoditization and it is indeed unagile, because the value of what you are doing has to lie in actual delivery. That is the only proof of success. The only acceptable commodity is a working product.

Even so, I felt that Dean was on to something. I have met the sort of board-room horrors he describes and is trying to “placate”, as Johanna put it. I had to give Ken the following reply:

If you make your estimates transparent, you can’t stop some damned fool from wanting to commoditize them.

I’ll even state that as a law of sorts, and unfortunately I’m not being entirely facetious. I think that the possible consequences of commoditization are genuinely frightening. History shows there’s no real limit to the complexity of exchange traded instruments, and there are even fewer constraints around over-the-counter deals. In a few years’ time we could see new exotics on the derivatives market, at least partly based on the numbers held up in planning poker sessions at hot-shot companies. Once estimates become commoditized - and some damn fool will eventually try - exactly that sort of nonsense becomes possible. It could make subprime look like the epitome of financial prudence.

Conclusion

What Dean Leffingwell seems to have done with SAFe is to acknowledge that commoditization is part of the toxic, background radiation of enterprise transitioning at scale. This management dysfunction clearly isn’t being challenged out of principle in the way that Ken or Johanna have set about doing. Instead, SAFe treats the commoditization of estimates to be axiomatic, a “given thing” that is germane to the transformation problem. The essential precept, perhaps, is to choose your battles wisely. SAFe’s advice seems to be: accept that commoditization is going to happen, address other issues that are more under your control, and make the best of this bad job.

Where does that leave us then? Well, when it comes to the matter of theory versus practice, I don’t know if Dean Leffingwell is right for his willingness to make such pragmatic compromises, or if Ken Schwaber is right for believing that certain core principles must hold in order to have a scalable agile framework that is worthy of the name.

Still, if I have to make a call…

Yogi Berra is right.