The Kanban Sandwich: A Bite-Size Recipe for Agile Work Flows at Scale

If you've been involved in Agile development for the past 10 or 15 years, you may have noticed that certain patterns of Agile application have emerged. The Scrum Patterns repository has captured at least some of them. Like the Design Patterns that were identified in the late 1990s, these patterns can be found in use across a wide swathe of the IT industry. The difference, of course, is that they relate to Agile development methods rather than to logical system specifications.

The identification of method patterns fascinates me, and I have found the number of corollary antipatterns to be at least as vast and no less entertaining. However, there is one good and reasonably clean Agile pattern at enterprise level that I've recognized and that I'd like to talk about here. I expect that most people reading this will be Agile developers — people who are familiar with the "coal face" of Agile delivery — so that's where I'll start.

Transitioning from project work to support

Every project must come to an end, otherwise it wouldn't be a project. That's true by definition; the point I'm making is that eventually a product backlog is exhausted or the money runs out. In an ideal project, the burn-down charts that an Agile team has dutifully provided will start to bottom out as the dregs of the requirements are drained. At some point any further work will not be thought to be worth the chase, and the project will be declared done.

Of course, that doesn't mean that the system ceases to be. It has to be handed over so the project team can be released and made available for other development activities. This handover could be to the support personnel of an external client, or it could be to a support team within an organization if the "customer" is an internal one. In a good, well-executed Agile project, this process of handover will have been done gradually and will have started as soon as a minimum usable subset of functionality became available. There should have been multiple releases into a production environment in accordance with a robust Definition of Done, and preferably by using a continuous build and deployment schedule. Less Agile implementations might only have deployed their "releases" as far as preproduction environments. They would then have to deal with the risk of unforeseen go-live problems (such as the payback of mismanaged technical debt) before the handover to support teams could be expedited.

This separation of the Agile lifecycle into project and support modalities gives us the first elements of a recurring pattern:

• • Project: The development and delivery effort, which is represented by a method framework such as Scrum, along with the associated de-risking of a product backlog by breaking it up into sprints and potentially releasable increments.

  • Support: "Business as usual" activities that follow development and delivery, such as common administration tasks or minor enhancements and defect fixes. These are small and repeatable changes, for which the quality of service may be allowed to vary (e.g., by service level agreement). This type of work constitutes the ideal diet of a Lean Kanban operation.

Let me be clear that these two modalities of Scrum and Lean Kanban are not meant to be stage-gated phases. As stated earlier, in Agile practice the transition from project to support should happen gradually, and it should begin as soon as possible. The transition should be continuous rather than discrete, and it can potentially amount to a gradual change in Agile practice by the same team. For example, it is not unknown for a project team — or a significant number of team members — to follow a system into production and maintenance. This is a mini-pattern in itself; over the course of time, a Scrum team can often find itself becoming more of a Lean Kanban one. The resulting hybrid process is sometimes referred to as "Scrumban," and the recent DevOps movement borrows much from this idea.

Meanwhile, up at the business end...

Agile developers are used to working with business-level requirements, such as user stories and their acceptance criteria. They're also used to working with business-level people such as product owners so that requirements can be turned into significant releases of business value. This can lead Agile development teams into the conceit of believing that they have an enterprise view of everything that is going on.

This belief doesn't normally hold water, because there is a great deal of work that has to be done before a team is put in place and its project gets off the ground, as well as work that's needed to keep the lights on while the project is underway. For example, contracts must be negotiated so people can be paid, project budgets must be proposed and approved, personnel must be resourced, equipment must be secured, and an understanding of the project scope must be arrived at so that a first cut of the product backlog can be elicited. If these things aren't done up front, the team will face a very long and frustrating Sprint Zero.

So who sorts this out, then? Well, that's where the second part of this pattern comes in: the use of Kanban at the program/portfolio level of an organization. But first let's look at what this level is, how the participants perceive things, and what their contribution is meant to be.

The program/portfolio level

There is a facet of an organization that lies beyond the horizon of most Agile developers, and that they can find difficult to appreciate. This is known as the program or portfolio level. All of the projects and BAU (business as usual) work that an organization performs are, or should be, part of a larger portfolio or program of work that reflects its strategic goals and ambitions. Agile teams, even at a Scrum of Scrums or release planning level, are by contrast focused on the operational delivery of project scope. Although terminology differs, the driver behind these very projects — the setting of their budgets and resources, their time constraints, and their business goals — is likely to be something called a "program board" or a "portfolio board."

Developers are justified in viewing this body and its work with a measure of healthy cynicism. The question is, how does a program or portfolio board keep its own house in Agile order and support the organization's Agile efforts in an Agile way? For many years the answer was a simple one: It didn't. Even if an organization was supposedly "committed" to enterprise Agile transitioning, there would be a glass ceiling that Agile practice would not be allowed beyond. Agility was positioned as something that delivery and operations teams should be doing in order to provide better value, but the very idea of senior managers changing their ways to support it would make them spit their cigars into their sherry and would likely earn the offending consultant who mentioned it a swift trip out of the door.

Much of this dysfunction and cultural malpractice remains in place today, especially in the public sector. However, a push-back against the ossification of senior management has started. The driver is that when Agile practice is implemented at a lower level, the impediments that emerge from higher up tend to become visible. The inescapable conclusion is that Something Must Be Done if shareholder value is to be improved. What though, and how? At least part of the answer lies in the translation of something called a "project pipeline" into fully-fledged Kanban.

The project pipeline

Projects don't appear out of a vacuum; they must be conceived of, validated, prioritized, and funded. The higher-ups who do this work have, for many years, represented this value stream as a "project pipeline."

In effect, a project pipeline is a kind of backlog, and it can be used in a broadly similar way to the backlogs developers are familiar with, although it will be managers who do the grooming. Anything that appears on it is inspected and critiqued by stakeholders. Another key difference is that each item is a potential project, not a user story representing a small piece of project value. These items may be correlated into epic-like groups or "investment themes." As the pipeline is groomed, some items will drop out of it entirely or will be shifted to the back of the pipe and kept on the back burner. Eventually priorities will be thrashed out and a schedule for delivering the projects will begin to emerge. If the higher-ups do this right, those projects that are given the go-ahead will tie in with the roll-off of personnel from projects that are closing out. I'm sure that many of you know about the pain that's involved when managers get this wrong, and developers are consequently expected do multiple jobs at once.

Now, I'm not trying to position this pseudo-backlog as evidence that senior managers have their own beautiful way of being Agile. For example, the transparency that Agile practitioners covet is not always there. The grooming of a project pipeline can involve a degree of Machiavellian intrigue and politics. Yet I also need to point out that the project pipeline isn't just a tool for suits in large blue-chip companies. I remember that during the late 1990s, many tech start-ups also used them as a way of organizing the highest-priority work that would generate revenue or raise capital.

Turning the project pipeline into a more Agile way of working

So, what we have with a project pipeline is a prioritized list of strategic requirements that can eventually become projects in the real world. As such, it is a candidate for becoming a genuine backlog if the supporting Agile mechanics and behaviors are there. What we have to do is make sure that this happens.

As always, the first step is to introduce transparency. An information radiator needs to be set up so that the work flow can be conceptualized, velocity managed, and work in progress limited. A Kanban board of some sort is ideal. The columns that should feature on the board are, of course, up to the program or portfolio board for which we are trying to encourage Agile team behavior. However, one suitable model that I have seen uses the following:

• 1. Agreed backlog

  2. Allocated portfolio

  3. Work In progress

  4. Working with capability BA's

  5. In engineering

  6. Post-implementation review

  7. Future idea

It isn't just business ideas that get fed into the portfolio Kanban; it's also important to feed architectural ideas into it too. This is critical if an organization isn't going to get too far ahead of itself and build up unsustainable levels of technical debt. In other words, each project has to be seen not only in the context of business value but also in terms of its architectural impact upon organizational resources in the longer term. All of these backlog items, regardless of nature or origin, will need to be prioritized and scheduled according to strategic goals. What is needed is a just-in-time approach to maintaining architectural runway. The architectural requirements should be met just at the point when the business needs them — not so late that the delivery of business value is delayed, and not so early that significant architectural rework needs to be done as requirements change.

What makes this Kanban rather than Scrum is the fact that there is typically only a single backlog. Project ideas can certainly be organized around epics and investment themes, but it is usually overkill to break these up into sprints; for one thing, each time-box would have to be expressed in terms of several months or possibly even years. Also, the establishment of work limits is of central importance — the organization cannot be allowed to bite off more than it can chew. Kanban is, after all, a closed economy; in the most efficient implementations, work cannot be added in until an equivalent amount of work comes out the other end and the corresponding resources are freed.

Putting it all together: The Kanban Sandwich

To recap, what we have then is a Kanban-based Agile way of working at a support level, in which small and repeatable changes are handled after a project closes. The project itself is run in Scrum fashion, with multiple sprints being used to de-risk scope by the early and continuous release of value. At the enterprise level we find another Kanban, where project ideas are fed into a backlog that is analogous to a traditional project pipeline but that can be improved to support more Agile practices in the boardroom. In short, it's a Kanban Sandwich with a Scrum-like process in the middle.

This pattern can be found across the industry, and something very like it features in the Scaled Agile Framework. Similar ideas are associated with the agility@scale approach, where such issues have been framed in the context of DevOps and a business-driven project pipeline that supports architectural visioning.