Is agile transformation at risk?

The production of complex systems is fraught with difficulty. Technology has become more complex and increasingly pervasive in our lives, and its development is now more likely than ever to incur technical debt. Organizations experience missed deadlines, increased rework, blown budgets, drastically "revised" scope, and ultimately a failure to deliver the value stakeholders expect. Plainly, the way modern systems are developed has not kept pace with other technical advances.

The great irony of our times is that the solution is well defined and well understood. The frequent, iterative, and incremental delivery of value is known to be the answer. By releasing value early and often, large projects can be substantially de-risked and timely planning adjustments can be made. Deadlines are more likely to be hit, rework reduced, budgets met, scope controlled, and in the end good value is far more likely to be provided. The great challenge has lain in getting organizations to adopt such a way of working.

For the past fifteen years the agile community has led this initiative for change. We can claim great success in having won the argument at an abstract level. Enter the IT department of any large organization, in either the public or private sector, and you are more likely than not to find lip service paid to "agility". Moreover, executive sponsorship for agile practice would seem outwardly to exist. After all every CEO or head of a government department would like their people to be "agile"...conscientious about the need to deliver value, and responsive to change. Who among these movers and shakers would dare to suggest otherwise?

Therein lies the real problem. Many organizations have taken the vocabulary and outward form of agile practice, but without sponsoring the deep change that agile adoption genuinely requires. They engage in shallow illusion, renaming existing roles or dividing a schedule up into equal units of time so iteration is faked. The benefits they hope for escape them...and their projects continue to fail. In short nothing really changes at all. It is left to other organizations, the ones which actually make the agile journey, to reap the rewards. There is no mystery to their success. Agile adoption is hard and it cannot be reduced to an exercise in sympathetic magic.

For many organizations then, agile transformation is at risk. In fact they face a double jeopardy. Not only do their projects remain hobbled by continuing bad practice, but the opportunity for change is lost. Hopes are raised and dashed, and it is the ignomy of failure rather than the sweet smell of success which is publicized. From Universal Credit and Healthcare.gov, to DMI, the Surrey Police System and dozens of projects in between, we see agile initiatives betrayed by weak implementation and followed by bitter disappointment. Reputations are damaged and the goodwill to repeat such experiments wears thin. When will another opportunity for change appear again?

It doesn't have to be this way. Big organizations can be as lean and as agile as small ones. Agile transformation is within the grasp of the large enterprise, if only the willingness to achieve deep change is genuine and the patterns of success are understood and applied. Once that is acknowledged, any company or government department can indeed become a “startup at scale”.