Agile Transitioning and Transparency: Let the Dog See the Rabbit

Well, in my view the first step towards a successful agile transition is the attainment of transparency. I regard a Kanban board, which exposes work as tickets (cards) and their actioning in response to certain signals, to be a good tool for encouraging this. It doesn't matter if a team harbours ambitions to follow Scrum or some other method. One of the first things I will do on a project is to raise such a board and make it visible. My goal at this early stage is just to expose the current state of play…who is doing what. I won't be trying to improve process at this point. All I'm after is a first-cut rendering of the board. Usually there's quite a lot that is supposedly being worked on (see first illustration). I'll then challenge this first attempt by identifying the actual work that people are doing, tentatively keeping this work "in progress", while also finding out what if anything is blocking them. I'll separate internal blockages from external ones, i.e. the impediments that a team can deal with itself, and those that require actioning by an external party. I'll also identify the work that people think they might be doing shortly - a first stab at a backlog. Other than that, I'll make no assumptions about process. The client could be following a waterfall scheme, or no process at all. The board (see second illustration, below) will have the following characteristics and constraints:

1. I'm interested in the work people are actually doing and expect to be doing, not just the work they are supposed to be doing or are thought to be doing.

2. Some of the work being exposed may be completely unrelated to assigned tasks. No matter…if it's taking someone's time, I want it to be visible.

3. There will be no restrictions to the number of cards that go on the board at this point.

4. The states (columns on the board) are "Backlog", "In Progress", "In Test" (or "In Review") and "Done". There will also be a "Blocked" column, though blocked tickets can simply be turned upside-down.

5. The team boundaries may be amorphous, with no clear indication of who's a member and who isn't.

6. The tickets are written as one liners…perhaps just two or three words, but which mean something to the people doing the work.

7. The tickets are not necessarily correlated to proper user stories with acceptance criteria.

8. There may be significant overlap in the work that each ticket represents.

9. One ticket may be substantially more complicated and time consuming than another, and this disparity may be unrecorded.

10. The Definition of Done is likely to be inadequate (e.g. "all code checked in"), or tautological (e.g. "testing completed").

Why this "transparency first" approach is controversial

Needless to say, there are no best practices being enforced at this stage. In Kanban terms there will be no attempt to enforce Work In Progress (WIP) limits, to encourage pull, or even to record velocity or other basic metrics. This is a controversial position to take, and arguably a heretical one. Many agile coaches, including some very knowledgeable Lean ones, advocate the embedding of good process as the first step to take in an agile transition. The use of an actual Kanban board comes second, if at all.

For example, Jim Coplien sees the recent fad for Kanban boards as a misappropriation of the Lean system originally promoted by Taichi Ohno. He believes that a Lean process, optimised around one-piece flow, needs to come first. Coplien observes that "We see teams adopting this [misappropriated] form of kanban, as a tool or methodology in its own right rather than as a worldview, without first having built foundations and disciplines of one-piece flow." When these best practices are embedded, Coplien argues, there may not even be a need for a Kanban board. He cites pair programming as a catalyst for achieving one-piece flow and for reducing the need for card-based transparency. "Good pair programming is quite unstructured", he says. "Because the feedback loops happen locally and immediately there is no need for a literal kanban card". I consider the theory behind this assertion to be sound. However I also value the adoption of a Kanban board purely as a tool for transparency, and not just as part of a systemic "worldview". I value it as a baby step towards achieving the wider gestalt of agile practice. You have to start somewhere…and in my experience at least, the key to achieving progress is knowing where you are now.

I am cognisant of certain exceptions. For example Jeff Sutherland can be seen as falling into the "Process Improvement First" camp with his Shock Therapy approach to Scrum transitioning, and I see the rationale behind it. I consider this approach to be analagous to the "total immersion" method for learning foreign languages. In my opinion it's a great technique…if you can get away with it. Unquestionably, Shock Therapy can provide fantastic results that would otherwise take years to achieve. The problem is that many clients - I would say most - just don't have the ability to absorb very much in the way of shock. This isn't necessarily due to a lack of mettle. An organisation in transition is rarely the whole organisation. Realistically, you can expect to be helping just a small part of the enterprise. You can expect to be helping people who are themselves constrained by dependencies upon non-agile departments that are indifferent to agile practice, or hostile towards it, or they might be dealing with other bodies that proceed at a glacial pace. Out of those first tickets that I put up on the board, ninety percent or more could be blocked because of such dependencies. Quite frankly, the discovery of that fact, and learning that they have to find a way to deal with it, is about as much shock as a typical client can handle. I therefore believe that the first duty is to expose what is actually happening…let the dog see the rabbit. That's where the early introduction of a Kanban board comes in. I want transparency first so I can show what is going on. Much though I'd like to get cracking on achieving better agile practice, process improvement needs to come second.

Preparing to improving the process

So then, what I will usually have at this point is an awful-looking pseudo Kanban board and no improvement to the underlying process. However, if I've done it right, any ugliness will be a reflection of truth.

Just to recap: usually a board will start with a mountain of work that is supposedly "In Progress". I've then challenged it on a ticket-by-ticket basis to find out what people are actually doing now. If they haven't started on a ticket, it gets moved to the candidate backlog. If it has stalled for some reason, it gets annotated with the constraint and marked as blocked. If there is a clear distinction between internal impediments and external ones, it needs to be represented accordingly - perhaps with labels, separate columns, or by using different areas of the board (see second illustration).

Once I have attained a basic level of transparency and shown what is going on, I can start improving on the process. The four steps towards achieving this are:

1. stabilisation

2. delineation

3. metrics

4. pull

These are to be followed in order, with some degree of overlap from one to the next.

Over the next few weeks we'll consider each one of these in turn. In my next post we'll consider the first of these, stabilisation, in which we try to reduce the level of flux between project components. We'll cover stakeholder identification and the normalisation of a team, the introduction of daily standups, and the refactoring of tickets so they are small and independent - a key step towards creating proper user stories. We'll also look at putting together a candidate backlog and risk-driven release train under the constraints of quality, cost, time, and scope.

Note: The Agile Journal became Techwell shortly after this posting. Further posts were made, to a revised agenda, in The Agile Zone.