Disruptive Innovation NetworkApply the Transformation FrameworkBuy the Book
Welcome to
> agilepatterns.org
Empirically managed agile transformation


Agile Development in Practice: the official reference book


Agile Development in Practice
The official agilepatterns.org reference book is now available. Comprising over 250 pages of material, including a foreword, 16 tightly focused chapters, and extensive appendices, this is a handy desk reference for anyone considering the agile transformation problem.

"The development of complex systems is fraught with difficulty. Many organizations have taken the vocabulary and outward form of agile practice, but without sponsoring the deep change that agile adoption genuinely requires. The benefits they hope for escape them...and their projects continue to fail. All too often, nothing really changes at all. This book tells you what agile practice is really about, and how to achieve it. The essential components of an agile way of working are laid out chapter by chapter. The book concludes with a concise treatment of the agile transformation problem, and shows how to leverage agile patterns and practices in order to resolve it."


 Agile Development in Practice is available from amazon.com.


Pattern of the Month

Backlog

Backlog
Intent
: Enumerate all work to be done in the form of an ordered list

Proverbs:
  • When everything is a priority, nothing is a priority
  • First come, first served
Also Known As:
  • Prioritized Requirements List
Motivation: Requirements can originate from multiple sources and stakeholders. As such, priorities are often unclear and the scope of changes may overlap, making estimation difficult. A single repository of ordered requirements is needed.

Structure: A backlog will consist of multiple backlog items, all of which will be placed in order. Each item will be sufficiently clear and unambiguous to be given a size. Items can be added or removed from the backlog, or re-ordered within it, a process known as backlog grooming. Grooming also allows backlog items to be modified if necessary.

Applicability: Backlogs and the associated principles of queue management are applicable to all agile methods. 

Consequences: A sized and ordered backlog allows estimates to be made regarding the likely time of item completion, assuming that the velocity of delivery is known. A less reactive management style is cultivated and “firefighting” is reduced. Backlogs require good management and clear ownership. Establishing this discipline can be a problem for many organizations.

Implementation: Scrum teams have two backlogs: a Product Backlog, which contains all product requirements as they are understood at the current time, and a Sprint Backlog which contains the work items needed to complete a selected portion of those requirements. DSDM has a Prioritized Requirements List which is analogous to a Product Backlog. Kanban teams generally only operate using one backlog, although the constituent items may require different qualities of service. 

See Also
:

Agile Transformation in the News


OCTOBER 3 2017: In a recent McKinsey survey into agile transformation, respondents who described their business environments as "unstable" were more likely to claim that their organizations had begun a transformation attempt. However, few such enterprise transformations would appear to have been completed. Only 4 percent of all respondents said that their organizations had fully implemented agile practice, while another 37 percent claimed that enterprise transformation is in progress.

SEPTEMBER 11 2017
:
ANZ Bank is restructuring to create '150 start-ups', according to the Sydney Morning Herald. Note: The bank is broadly attempting to emulate an approach which has come to be known as the Spotify model.

AUGUST 22 2017
:
IBM has said it will use Agile approaches not just as a methodology for developing software but as “an engine for business transformation”. Diginomica reports that to this end, IBM has spent $380 million in setting up a number of “agile hubs” in six strategic centres in Austin, Texas, San Francisco, Atlanta, New York City, Cambridge, Massachusetts, and Raleigh, North Carolina. Note: The need to improve collaboration and to Fail Early Fail Fast are evidently significant factors.

AUGUST 3 2017
: Software created in corporate "innovation centers" often fails to make the break into mainstream development. In this CIO Magazine article, TD Ameritrade CIO Vijay Sankaran explains how he intends to break that cycle. Sankaran says he has turned to agile practices, design thinking and lean startup methodologies for quickly building adaptive products. "There is an opportunity to leverage digital technologies to continue to disrupt" he claims. "If you don't do the devops piece in parallel where you have automated testing and continuous integration and you can deploy packages automatically you create a bottleneck in a different place". Note: Sankaran says this has allowed the innovation center to deliver prototypes in one to three months and minimally viable commercial products in only six months.

JUNE 21 2017
The Department of Homeland Security has canceled a large-scale project to make agile software development services easier for agencies to buy, reports Nextgov.com. The initiative let agencies depart from the old acquisition process which could take months or years to work through bids. However, the new system proved too complex to be viable, with 100 companies participating in half-day technical challenges which had not been adequately prepared for, resulting in a "logistical nightmare". Bidders who didn't make the cut were then in a position to claim that the acquisition process was unfair.

MAY 3 2017: The UK 'wastes billions every year' on failed agile projects, according to this article in IT Pro. It reports that "More than half of CIOs think the methodology has been discredited and is just an IT fad". Reference is made to a survey in which it has been found that "Just under one third (32%) of projects are failing because the teams are too geographically dispersed, while 34% have failed because the teams didn't plan before getting started or didn't plan sufficiently as the project developed". Note: Geographical dispersion is a symptom of the unbounded team antipattern while a failure to prepare suggests uncommitment on the part of stakeholders who may be too busy. Inspection and adaptation underpins an agile way of working, where replanning ought to occur on an ongoing basis.

APRIL 3 2017: The Register reports that "hundreds of millions of pounds" have been wasted on a supposedly agile delivery program for digitizing the UK's judicial court services. "Particular scorn has been directed at the so-called 'agile experts' who have been in charge of managing the programme. 'There is no plan, no artefacts, no direction, just constant excuses,' said one insider. 'How they can still be in place as well as still being allowed to recruit 'experts' with absolutely no delivery after 30 months is scandalous.'" The Register also cites an insider as saying that "if the programme were following proper agile principles, by this stage there ought to be at least a dozen meaningful services available across the criminal justice system". Note: A failure to plan and release value incrementally is contrary to an agile way-of-working, as there can be no empirical evidence of progress. The result in this case has been an escalation of commitment when controlled failure may have been the wiser option.

MARCH 1 2017: Eastern Foundry, a collaborative workplace for Federal Government contractors started by military veterans, has spoken out in defense of the 18F initiative. Writing in Nextgov, they point out that although the 18F initiative has struggled, "there is a cost to opposing innovation and forcing reform to move at the pace of government acquisition...18F was created out of a massive procurement failure, and it has moved mountains to help agencies and the nation avoid such costly errors". 

FEBRUARY 17 2017: A report by diginomica asserts that although growth has been modest, large enterprises are nonetheless buying in to the agile development approach. In this article Jon Reed looks at Computer Economics’ new survey on agile development, and gains insight into the continued relevance of agile practice and why barriers to adoption persist. Note: Two essential patterns are elicited as a takeaway: "Agile is really about iterative cycles. When you combine agile with a minimum viable product approach, you have a huge edge building relevant software that customers/users actually feel like using".

JANUARY 26 2017: Mobomo, a Washington DC supplier to the U.S Government, has explained its use of agile techniques in the Federal space. In this interview in Application Developer Magazine, they explain the key roles and benefits and the new disciplines which are needed. Note: Many good patterns are identified here, including Product Ownership, Teamwork, Servant Leadership, and Swarm.

JANUARY 12 2017: A new CA Technologies (NASDAQ:CA) global study indicates that there are great advantages to be had in using agile and Devops techniques together. Mature adopters "realized significant increases of up to 52 percent in customer satisfaction and up to 50 percent in employee productivity", according to this report in BusinessWire. Note: Those advanced users recognized that agile and DevOps practices have to be used jointly before benefits can accrue. Just as DevOps is about changing organizational culture - and is not simply a matter of automation - so agile delivery requires teamwork that bridges the "development-operations divide" and optimizes the value stream.

DECEMBER 20 2016: Software development and operations teams working for the Federal Government need better collaboration and use of automated testing. Writing in CollabNet, Thomas Hooker says that "Agencies using traditional Waterfall methods for software development will need to strongly consider shifting their strategy to Agile practices in order to scale and streamline the software delivery lifecycle". The driver is new standards for compliance, governance, and security. He says these demand that "the bar should be set high when it comes to the security and quality of technology". Note: Many agencies struggle to adopt agile practices and resort to the Waterscrumfall antipattern rather than sponsor the deep and pervasive change which is required.

NOVEMBER 10 2016: The digital recruitment head at the Department for Business, Energy and Industrial Strategy (BEIS), has resigned. In a damning blog post picked up and reported by Government Computing, Ann Kempster criticized "...a lack of appetite in departments for real, meaningful transformation. I’m seeing a lack of effective leadership right from the very top of the civil service...There is a lack of vision, lack of ambition and lack of any sort of a plan anywhere". Note: The agile transformation crisis will continue in Government until appropriate sponsorship and vision is in place for the deep and pervasive change that is clearly needed.

NOVEMBER 3 2016: Bipartisan congressional support for the Modernizing Government Technology Act of 2016 (MGT Act) has improved efforts to modernize federal IT, according to Claire Bailey in InsideTechTalk. Concurring with Federal Computer Week that "modernizing big federal IT systems isn’t really about the technology—it’s about solving user’s problems effectively", she says that the need to actively involve customers in agile practice - and to be prepared to fail early and fast - are important steps. Note: Effective Product Ownership underpins successful customer liaison, especially at scale where many stakeholders can be involved. The Controlled Failure pattern should also be mastered if the Death March antipattern is to be avoided.

JULY 18 2016: The Department of Homeland Security agency has only partially implemented an agile way of working, a congressional audit has found. While the Citizenship and Immigration Services department claimed to have adopted agile software development techniques for its huge, $3.1 billion technology transformation from paper to digital, success has been patchy. According to an article in FedScoop, User Story prioritization has been accomplished reasonably well, but integration of deliverables has been poor. Note: The timely release of fully integrated increments of release quality underpins agile practice. Without this the value stream will be inhibited and release orchestration will be compromised. Note also that the audit itself set an expectation that release planning ought to be complete prior to initiating development, which is itself contrary to the agile principles that are supposedly being adopted.

MAY 12 2016
: The BBC has been criticised for thinking that agility means "making it up as you go along". According to an article in The Register, the National Audit Office found significant shortcomings in the organization's project for customising the user experience of the MyBBC portal. “MyBBC is an ‘agile’ project that was designed to define benefits as the project progressed,"  the NAO is quoted as saying, "but two years into the project it still was not clear what the BBC expected MyBBC to achieve overall”. The report goes on to claim that benefits were only defined late in the day, two years after the project had started. Note: The BBC Trust is quoted as saying that the project "did not define its its expected benefits upfront" because it was "an ‘agile’ project where benefits were to be defined as the project progresse[d].” This is a misunderstanding of agile practice, since a Minimum Viable Product ought to be delivered regularly and incrementally, where each MVP is framed in order to validate or invalidate a clear and testable hypothesis. Failure to do so can encourage Vanity Metrics, an antipattern in which success is not measured objectively, but rather by attributes that show the initiative in the best light.

MARCH 8 2016:  There is no linear relationship between the number of validated hypotheses and a team’s subsequent success, according to an article in Harvard Business Review. Ted Ladd provides thought-provoking evidence that although "the Lean Startup method works", and that validated learning helps teams perform "almost three times better in the pitch competition than teams that did not test any hypotheses", more is not necessarily better. In essence, he has determined that the relationship between validated learning and subsequent success is non-linear. "There was no linear relationship between the number of validated hypotheses and a team’s subsequent success" he observes. "I also found that teams that conducted both open-ended conversations and more formalized experiments with customers actually performed worse in the competition than teams that conducted either one or the other during the early stages of venture design". Note: The author has cited possible explanations for this such as erosion of confidence due to rapid changes in direction, or managers running out of patience for continued testing. Both are plausible. Validated learning may be applied as an enabling constraint, but not as a substitute for having a sponsored vision.

JANUARY 25 2016: CompuWare CEO Chris O’Malley has expressed optimism that mainframe development can bridge the chasm to agile practice. The main challenge he sees is changing the culture. “The gravity against change is huge”, he said. He explained that cultural change is hard because team members will always come up with a "rational" reason that their particular area can't change…even if they see the necessity of change for the rest of the organization. Note: The application of the Transformation pattern requires clear executive sponsorship so that deep and pervasive change can be effected across an enterprise. Employees must be left in no doubt that their own behaviours and working practices are expected to change too.

JANUARY 6 2016: According to Hewlett Packard, just one year after its birth the fledgling Impact Hub Berlin "was a single email away from closure". The Berlin team had to accept they'd failed to implement their own model: "We teach all the time that we should prototype and we didn't even prototype", admitted CEO Nele Kapretz. The trick they used to recover was to "think like a scientist and treat every iteration of your idea as a chance to test a hypothesis". Note: this application of the Controlled Failure pattern saved the business by closing the validated learning loop in a timely manner.

Earlier stories from the news archive[...]

Recommended Reading

DevOps Studio model

The agilepatterns.org Briefing Notes. In each briefing an item of topical interest is brought into focus for executive sponsors and enterprise change agents.

Spreading Scrum through the Enterprise. A well-rounded look at the need for enterprise agile transformation and at the challenges organizations face when scaling Scrum. “Your competitors in your industry are disrupting the industry, so you are either on the disrupting wave or you will be shattered by the disruption wave.” Read more at SD Times.

How to Make Agile Work in Fast-Growing StartupsLooking to take the Agile plunge with your startup? Read these six practical lessons on how to make the transition easier and worthwhile. By Stefan Wolpers. Read more at The Agile Zone.

Beyond Agile Operations: How to Achieve The Holy Grail of Strategic Agility. Most organizations today are still struggling to master agile practice at an operational level. In this article, Steve Denning explains that, although developing these competencies is important, it is agility at a strategic level which must ultimately be achieved. This involves the disruptive creation of new markets where the agile organization brings value as an innovative network of players. Read more at Forbes Magazine.

The Lean Enterprise: Introduction to Massively Disruptive Innovation. At scale, lean organizations have cross-functional teams, culture in which all teams own the product, and embrace an innovative mindset. Failure is not only expected, but encouraged. Read more at The Agile Zone.

The Lean Startup Security Guide. A well thought-out set of measures and safeguards tailored to the security needs of early-stage startups, and which provides a solid foundation for growth. Read more at leanstartupsecurity.com.

Cultivating Disruptive Innovation in the Enterprise. A look at how innovation "colonies" (also known as innovation laboratories) help organizations to develop an enterprise lean startup capability. Read more at SolutionsIQ.

5 Common Pitfalls of #Agile Transformation in the Government (and how to avoid them). A brief examination of some of the pathologies that are most frequently encountered, including resistance to change, failure to deliver software frequently, siloing of work, lack of Product Owner empowerment, and the putting of politics before process. Read more at Coveros.

To Lead a Digital Transformation, CEOs Must Prioritize. Transformational leadership is about creating an agile organization that can detect what type of change is essential and respond quickly with the most competitive solution, says Laurent-Pierre Baculard in Harvard Business Review. The problem however, is that that these efforts tend to be ad-hoc and uncoordinated. CEOs need a holistic view of the digital threats and opportunities facing key parts of the business, and a way to link them to an overall vision. Read more at Harvard Business Review.

How to get agile to work at your company. The need for clear sponsorship in an agile transformation is highlighted in this article from CIO Magazine. Transformation requires a shift in thinking for all parties involved and the full support of C-suite leadership, the business and developer teams. Much of this shift is cultural, especially when people are more familiar with stage-gated waterfall practices. "Getting everyone aligned with how the new process will work is a challenge", the article observes. "In addition, some people have misconceptions and/or lack of understanding to what agile really means." Read more at CIO Magazine.

Agile in the UK Government - An Insider Reveals All. How much, if anything, have GDS achieved? Are £12 billion IT disasters still likely to happen? Key takeaways from this exposé include:
  • GDS are trying to transform the UK government, to become digital-by-default
  • GDS philosophy puts users first, with a focus on agile principles and practices
  • Continuous delivery is happening in many government agencies, including HMRC
  • Lots of UK government code is now open source
  • Big progress by GDS, but still lots of big challenges

Read more at InfoQ.

Why Agile Fails in Large Enterprises. According to one survey, 68.6 percent of teams have been involved in a project that they knew would fail from the start. Agile projects achieved a 71.5 percent success rate, compared with only 62.8 percent of traditional methods. In this article, Sanjay Zalavadia looks at some common areas which impact the adoption of agile development processes in large enterprises. Read more at InfoQ.

12 Failure Modes of an Agile Transformation. Among Jean Tabaka's many contributions to the betterment of agile practice was this article, in which she identified “12 Agile Adoption Failure Modes”. "It’s imperative that we look not just at Agile adoption, but at Agile transformation — where organizations move beyond Agile principles within their IT groups to business agility", she said. "To accomplish this, we transform from just doing Agile to being Agile". The failure modes she identified include a lack of executive sponsorship and a failure to transform leader behaviors. Each one of these modes can be viewed as an agile antipattern that inhibits successful agile transformation. Read more on the Rally website.

Scaling agile at Spotify
Scaling Agile at Spotify with Tribes, Squads, Chapters, and Guilds. Spotify, a Swedish startup launched in 2008, now has hundreds of agile developers divided into 30 agile teams spread over 4 cities in 3 timezones. Its approach to achieving agile practice at enterprise scale is widely regarded as a definitive case study in the matter. In this 14 page whitepaper, Henrik Kniberg and Anders Ivarsson explain the Spotify system in terms of team responsibilities, roles, and dependencies. Read more at Tech Crunch, including the implications for Enterprise Lean Startup.

How PayPal rallied a 4,000-strong move to Agile. PayPal’s wholesale move to an Agile methodology was built on ‘four pillars,’ took seven months and changed the way 4,000 IT and product people did their jobs. Read more on TechTarget about how it got done.

How Pairing & Swarming Work & Why They Will Improve Your Products. Swarming is a behavioral pattern that many teams would like to apply, but which they find difficult to implement. In this article Johanna Rothman compares and contrasts swarming with pairing, why both of these techniques work well, and how they ultimately help to improve product value. Read more at The Agile Zone.

Unspoken Agile Topics. Companies often decide to introduce Agile practices without thoroughly thinking through why they are doing it. For such companies, Agile adoption has almost taken the form of a fashion statement, a way to prove to themselves that they are up to date with others. In this paper Gene Gendel of Global Investment Bank looks at the impact of these problems on agile transformation, how it is measured, and at how it is implemented in terms of team roles and best practices. Read more at scrumalliance.org.

Why You Should Limit Work In Progress. "Sometimes I sit down at the end of a day or even a week and am dismayed that I don’t seem to have really accomplished anything. I know I was very busy and I know I was working hard on important things, but why don’t I have anything to show for it?" Read more on the importance of Limited WIP in this article by David Hammerslag at BigVisible.




We Support RetroMatproAgileagile north east




thecitysecret academyscrum.org professional training networkscrum.org