Good Housekeeping

I once worked on a very large software project. Over time, our bug list grew larger and larger, until everyone realized that something had to be done. A team was dispatched to clean up the bug list; to either fix the bugs or remove them from the list. A couple of months later, management announced with great fanfare that the bug list had been reduced from around a thousand bugs to some six hundred, thus reducing the number of bugs from astronomical to merely unmanageable.


... as a team develops the product, issues are invariably detected. There is a strong tendency to write issue reports and put these reports on an issue list to allow for classification and prioritization but doing this creates more work and does not resolve the issues.

✥      ✥      ✥

A Scrum Team may deliver only PBIs that are Done. Deferring the finishing touches on a PBI causes work to add up and having loose ends leads to confusion about the state of progress.

New development is more fun than fixing issues and fixing issues is disruptive — and not fun — so most developers don’t like to do this work. Therefore, there is a common tendency to park issues in a list of things to do after all the fun stuff is done. Collecting such loose ends and issues in to-do lists and backlogs creates extra administration, and because such lists can be out of sight and out of mind, information that supports and guides the day's work in progress is likely to disappear at least temporarily.

In a Sprint the Scrum Team members have defined their Sprint Goal and are focusing on achieving it. Issues that are reported are put on an issue list because the developers who worked on the PBI are busy with other tasks, and shouldn’t be interrupted. Issues that arrive might also be considered to have low priority, and it’s nice to have a place to put them. Issue reports and lists give people a convenient way of logging the issues. But it's easy for developers to leave them on the issue report list. (Maybe they hope that if they leave them there long enough, they will go away.) Such a list introduces waste and inconvenience to users. Each issue has to be handled at least twice, once when it arrives, and later when it is fixed. Issue lists are generally prioritized, but how do their priorities match against the priorities of development items? Keeping separate lists causes this question to be revisited constantly. This creates work and causes context switching for the developers. The longer an issue sits before it is fixed, the greater chance there is of losing information about the issue, how to reproduce it, and how to fix it. If issues are allowed to sit on an issue list, further development might even depend on the issues, making them that much more difficult to fix.

Bob Martin relates a story of once fixing a spelling error in an application. However, many customers had built screen capture scripts that depended on the misspelled word. He was ordered to “unfix” the misspelled word.

Written issue reports are designed to capture the problem, but they often miss important information, especially about how to reproduce the issue. The original finder of the issue may forget, or worse, no longer be available. And good issue reports take time and effort to write, both of which may be in short supply for a frustrated finder of an issue. Issue lists tend to grow with issues added faster than being resolved. If the lists are allowed to get large, it virtually ensures that the lowest priority issues will never get fixed (creating a FISH queue, First In Still Here). Issue lists usually contain user-visible issues. This creates the possibility that another user finds the issue as well. If the second user writes an issue report, you have duplicates, and managing duplicate issues is simply messy. And a user who finds a known issue is needlessly inconvenienced.

The Sprint is planned with Product Increment(s) that create value for stakeholders. Adding new features increases value, but fixing issues does not apparently increase value. So there is pressure to work on new features rather than fixing issues. If the team has a fuzzy Definition of Done, work might be considered Done, with some pieces deferred until later. Likewise, if the team's commitment to Done is weak, the same thing will happen. The work not done yet is basically an issue that is deferred.

In short, to defer issue fixing is to borrow against your future velocity. This puts the team in debt to the product in that the team owes the product work. We call this technical debt. And it gains interest over time!

Therefore:

Fix all issues in less than a day. Aim to have a completely clean product at the end of every day.

This is simply practicing good housekeeping. If you find a cockroach in your house, you kill it right away.

In order to make this work, avoid the temptation to keep an issue list. You already have a list of work to do, the Sprint Backlog. As issues are identified, they are either fixed immediately or go on the Sprint Backlog to be fixed later in the day or the very next day. This allows developers to complete their current task without interruption, if necessary. Naturally, if it's an issue that you find yourself, fix it now while it is fresh in your mind.

✥      ✥      ✥

Daily attentiveness to progress helps the team to learn how to produce a predictable Regular Product Increment.

For the sake of visibility, it's a good idea to put the issue on the Sprint Backlog or at least one of the other information radiators like the Scrum Board. Still, the ideal is to break flow and to resolve the issue immediately, rather than using the information radiators to schedule the repair. The team should establish the discipline that repairing defects is always highest priority, because defects add drag on velocity, may hide other defects, and tend to be forgotten if deferred.

If an issue cannot be fixed within one day, then that may indicate a more serious problem with the product. If the repair interval crosses some pre-determined threshold (usually a half-day or a day) of effort, the issue should be escalated via the Scrum Emergency Procedure. Causes of such issues should be discussed in Sprint Retrospectives.

It is usually necessary to have an issue reporting mechanism for customers to route issues to the tem. If you have one, use it only as a holder for issue reports that have not yet been examined. Once you examine an issue report, handle it immediately as described above. Resist the urge to read it and leave it in the issue reporting system to be handled later.

Following this pattern will help velocity in many ways. Good Housekeeping reduces the overhead of handling issues because teams don’t need to devote effort to maintaining a list of issues and periodically reviewing them and prioritizing them. Each issue is handled just once. The pattern also basically eliminates duplicate issue reports, which are cumbersome to handle. The Development Team will only have one list of work to do (the Sprint Backlog) rather than two (i.e. an issue list and Sprint Backlog), making it easier for the team to focus. It keeps shared work products clean. This consistent quality is probably the major contributor to the improvement in velocity. And the pattern eliminates the periodic “bugathon” or “bug bash” that can happen in software products (this is where the team stops what they are doing to spends time resolving outstanding issues, often with the goal of decreasing the number of issues rather than improving quality). Not only do these issue-hunt activities take time, but they can also breed other issues because the high intensity issue fixing induces fatigue and encourages one to cut quality and verification corners.

If a team already maintains an issue list, then moving to this pattern will be painful and require a lot of work. However, fixing issues immediately can reduce the time needed for testing. Since testing is often a delivery bottleneck, deployment will go faster. For these reasons, it is good to immediately fix issues discovered in a Sprint. The Developers and Product Owner together should develop a plan to reduce latent technical debt caused by past unfixed issues and undone work.

This is a radical break from the traditional way of thinking. Teams that maintain an issue list must not only eliminate their list, but must change their way of thinking. Strong commitment from the team is needed. Undertaking Good Housekeeping has an interdependency on the Scrum Team having a Definition of Done and the DoneMaster enforces this definition. The team may also require the ScrumMaster to be a Sheepdog to help remind developers to follow this pattern until it becomes a habit.

Good Housekeeping is an indicator of the Development team’s cross functional behaviors. Where team members are focused on working together to create a Regular Product Increment this practice is easily adopted. The contrasting case is where team members treat testing, issue reporting, or product quality as other people’s problems. In this situation adopting this pattern will be more challenging as the team members need to change from their traditional organizational behaviors to the cross functional behaviors required for Scrum.

The source of data on this is a paper on Scrum and CMMI [1] where a CMMI Level 5 company drove the doubling of productivity on every time using control charts that manages issue fixes to less than a day (ideally averaging two hours).

Note that while the focus of this pattern is on fixing issues some teams working on software products may want to apply this to refactoring. After all, refactoring is a way of cleaning up issues in the code. This is a mistake, refactoring needs to be part of standard development practice not treated as a separate piece of work to do. Deferred refactoring leads to technical debt so refactor as you write.

This pattern describes how Scrum Teams should manage issues that arise within the Sprint. Preexisting issues, should there be any, should be prioritized by the Product Owner and managed in the Product Backlog.

Coplien and Harrison's pattern Completion Headroom can be used to set aside time to fix issues. [2] Furthermore, the team can handle serious user-affecting issues using Illegitimus Non Interruptus.

Good Housekeeping is not only cleaning up your own stuff but also that of others. Robert Stephenson Smyth Baden-Powell, the father of Scouting, stated: “Try and leave this world a little better than you found it.” Today, Scouts say, “Leave the campground cleaner than when you found it.” [3] The same applies to leaving the product and shop floor cleaner than they were when you started. This approach is a significant part of the continuous attention to technical excellence that is a principle for agile teams. [4] You can help achieve Good Housekeeping with Whack the Mole.


[1] C. Jakobsen and J. Sutherland, 2009, Scrum and CMMI – Going from Good to Great: are you ready-ready to be done-done?, proceedings of Agile 2009, Chicago

[2] Coplien and Harrison, “Organizational Patterns of Agile Software Development,” Section 4.1.10, “Project Management Pattern Language,” Completion Headroom. Pearson, 2004.

[3] Lord Baden Powell, cited at http://www.biographyonline.net/humanitarian/baden-powell.html.

[4] Manifesto for Agile Software Development, 2001, http://agilemanifesto.org/principles.html , accessed 23 January 2017.


Picture from: Presentermedia.com.