Agile Beliefs

Dancing with Uncertainty

Change is a Competitive Advantage

In chapter XXX, we saw that unexpected requirements changes lead to uncertainty. Since uncertainty is scary, many methodologies try to make uncertainty go away by predicting all project requirements up front, creating a detailed plan based on them, and then controlling the work of the folks on the project by enforcing that plan. The approach does help deal with anticipated requirements changes but in doing so it:

  • Deliberately ignores and therefore magnifies the risks of unanticipated requirements changes

  • Deliberately delivers software late in the project, when the business value gained may be much lower than when the project started
  • Stifles creativity by controlling unpleasant surprises in a way that also prevents pleasant ones
  • Unintentionally leads to internal software architectures that become increasingly un-maintainable and eventually technically bankrupt

These problems might have been acceptable in slower, more predictable times. In the modern business world, though, change is much faster paced than it was twenty years ago. Companies can’t afford to be held back by their computer systems, allowing their competitors to go bounding ahead.

To keep up with the pace of change in the business world, smart companies have abandoned naïve attempts at complete predictability and instead have learned to live with and work with potentially high levels of uncertainty and change. What they have found is that they can work with change rather than fight against it, and ultimately use change to their competitive advantage.

In this chapter we shall see that we can indeed both have our cake and eat it; we can control risk yet also embrace unexpected change. This, however, requires discipline; not the discipline imposed by a dictatorial, bureaucratic old-style methodology, but self-discipline supported by a rigorous approach that guides and encourages creative effort amid constantly changing needs.

Gun Slingers of the Wild West?

A few months ago, I came across a team that was writing software for third world countries. Since the software needed to have a low retail price, they needed to produce it cheaply. This meant they had to go fast. The team delivered a new release of the software every two weeks. A manager in another team at the same company was horrified – this went against the grain of his own once-a-year delivery cycle. He told me that those guys were cowboys – acting like gunslingers in the Wild West. He never actually worked with them, but he was sure they were completely unguided, pursuing their own chaotic whims, resulting in unmanaged and unmanageable catastrophe. So, I went to visit the team leader, and found their practices to be almost the opposite of the manager’s suspicions. They had strong leadership, with very clearly defined practices, and high levels of continual discipline. As one of the team members said to me “we learned that you can’t go fast for long if you stay sloppy”.

If you fear that the alternative to predict, plan, and control means going back to the Wild West then you are going to be suspicious of claims from teams working differently. If they aren’t doing it your way, they must be doing it the sloppy way. Sloppy means careless and careless means risky.

Agile folks say that the biggest risk of all is not delivering software that is valuable to the business. Everything, then, should contribute to delivering valuable software. If you forget that, and focus instead on just following the rules of predict, plan, and control for their own sake, you risk descending into an abyss of bureaucracy that saps the very life from a project. Agile folks shun bureaucracy. Instead, they have found a sweet spot that keeps pace with the needs of the modern business world, and delivers high value software fast and often. They don’t do this by going sloppy. Agile, if anything, is even more disciplined than earlier approaches – but it focuses that discipline in different places and applies it in more productive ways.

Expect and Respond to the Unexpected

Up-front prediction of complete, accurate, and essentially unchanging plans is rarely achievable. The plans you produce will be ineffective, and so will the results you get when you try to enforce them. Except for in very well understood problem areas, effective plans are not the result of up front planning (other than in very sketchy terms). Rather, they evolve in response to changing conditions along the way.

This does not mean, though, that we simply abandon all discipline and wing it as we go along. Various originators of the Agile approach spent years gathering empirical evidence in the trenches, showing what has worked in the real-world in hundreds of successful projects. They found that those projects followed practices very different from those preached by popular pre-Agile methodologists.

What they found was that whereas pre-Agile methodologies put faith in highly defined repeatable processes, with people as substitutable role-fillers, successful real-world projects tended to have put faith in the people themselves as the best way to handle the unexpected things that will continually pop up throughout a project of any size. The process and the managers are there to guide people, not to control them, and it is left to the skill and judgement of the team members themselves to determine when the process is and is not working, and when it needs adapting to support the project and the team more effectively./p>

Shared Beliefs

I have been deeply immersed in a whole bunch of Agile projects at various organisations for close to ten years now, and have studied and tried out most of the documented flavours of Agile. What I have discovered from all of this is that Agile folks tend to share some beliefs than pretty much turns upside down the beliefs, described in chapter XXX, that pre-Agile folks tend to have. These shared Agile beliefs are:

  • Put faith in the people: Unlike the pre-Agile methodologies, which put their faith in detailed rules at the expense of people, Agile puts the people themselves back in charge. Agile tends to be have few rules and rituals to be followed slavishly. Instead a small number of heuristic principles emphasise guided self-discipline while a gelled team of capable people works together to find creative ways of solving problems.
  • Focus on creating software: They de-emphasise producing documents for their own sake, and focus instead on creating working software as the main product, viewing all other artefacts as only valuable if they contribute to the delivery of that working software.
  • Focus on face to face collaboration: Rather than assuming that frozen documents can capture and communicate all important project information, Agile folks are expected to communicate face to face. They coordinate and steer their own work and share knowledge, and concerns by continually discussing and evaluating progress with one another. The result is continually up to date and shared mental models of what the project is about, where it is now, and where it needs to go next.
  • Expect things to change: Rather than expecting requirements to be pinned down up front, they expect requirements to evolve over time alongside changes in the business. They keep customers involved along the way to assist with continual elaboration of those evolving requirements, and build on what they do know to learn about what they don’t yet know. Rather than expecting completely up front design, the design evolves throughout the whole course of the project, as requirements change and the project team discovers and invents appropriate and effective design changes in response. To ensure that requirements are up to date, and currently valuable to the business, a project is addressed a small chunk at a time, with requirements fleshed as far as possible (but no further) for the current chunk, which is then developed into working software in a short burst of highly collaborative work. At the end of each short chunk, a demonstration of the software is usually given to the customer, progress thus far is reviewed, adjustments are made, and the team refocuses on re-negotiating requirements for the next chunk of work in the overall project.
  • Accept that software is soft: Agile folks are not afraid to rewrite things that were previously thought to be finished when changing requirements call for even radical rewrites. To get over any fear that this introduces risks, Agile focuses hard on increasing quality and reliability throughout the whole of a project. In particular, rather than leave testing and delivery to the very end, there is a strong emphasis on early and continual testing, demonstration, and deployment to give early feedback and allow for immediate correction where necessary.
  • Encourage self reliance: Managers and customers are expected to give strong and continual input in terms of requirements, and to share their satisfaction with deliverables. The actual project teams has considerable leeway in how they solve problems they face along the way when satisfying those requirements and responding to that feedback.
  • Are reflective: Unlike fully defined repeatable methodologies, there is an undercurrent of ongoing evaluation not just on progress of the project, but also of the effectiveness of the specific practices being followed. At the end of each chunk of work, the team reviews and tailors their practices to the emerging needs of the team, the next project chunk, and the general direction in which the project as a whole looks to be heading. Agile folks believe that without such continual adaptation, there is a major risk that the methodology itself would become irrelevant to the unexpected and emerging project needs that unfold as the project progresses.

Book

This is part of a book I am writing. More will be unfolding here and on my blog. Ongoing feedback would be much appreciated.