> agilepatterns.org
Agile Development in Practice: the official reference book
The official agilepatterns.org reference book is now available. Comprising over 250 pages of material, including a foreword, 16 tightly focused chapters, and extensive appendices, this is a handy desk reference for anyone considering the agile transformation problem.
"The development of complex systems is fraught with difficulty. Many organizations have taken the vocabulary and outward form of agile practice, but without sponsoring the deep change that agile adoption genuinely requires. The benefits they hope for escape them...and their projects continue to fail. All too often, nothing really changes at all. This book tells you what agile practice is really about, and how to achieve it. The essential components of an agile way of working are laid out chapter by chapter. The book concludes with a concise treatment of the agile transformation problem, and shows how to leverage agile patterns and practices in order to resolve it."
Agile Development in Practice is available from amazon.com.
Intent: Allow unplanned work into a timebox.
Agile Transformation in the News
JUNE 21 2017: The Department of Homeland Security has canceled a large-scale project to make agile software development services easier for agencies to buy, reports Nextgov.com. The initiative let agencies depart from the old acquisition process which could take months or years to work through bids. However, the new system proved too complex to be viable, with 100 companies participating in half-day technical challenges which had not been adequately prepared for, resulting in a "logistical nightmare". Bidders who didn't make the cut were then in a position to claim that the acquisition process was unfair.
JANUARY 26 2017: Mobomo, a Washington DC supplier to the U.S Government, has explained its use of agile techniques in the Federal space. In this interview in Application Developer Magazine, they explain the key roles and benefits and the new disciplines which are needed. Note: Many good patterns are identified here, including Product Ownership, Teamwork, Servant Leadership, and Swarm.
MAY 12 2016: The BBC has been criticised for thinking that agility means "making it up as you go along". According to an article in The Register, the National Audit Office found significant shortcomings in the organization's project for customising the user experience of the MyBBC portal. “MyBBC is an ‘agile’ project that was designed to define benefits as the project progressed," the NAO is quoted as saying, "but two years into the project it still was not clear what the BBC expected MyBBC to achieve overall”. The report goes on to claim that benefits were only defined late in the day, two years after the project had started. Note: The BBC Trust is quoted as saying that the project "did not define its its expected benefits upfront" because it was "an ‘agile’ project where benefits were to be defined as the project progresse[d].” This is a misunderstanding of agile practice, since a Minimum Viable Product ought to be delivered regularly and incrementally, where each MVP is framed in order to validate or invalidate a clear and testable hypothesis. Failure to do so can encourage Vanity Metrics, an antipattern in which success is not measured objectively, but rather by attributes that show the initiative in the best light.
Earlier stories from the news archive[...]
Recommended Reading
5 Common Pitfalls of #Agile Transformation in the Government (and how to avoid them). A brief examination of some of the pathologies that are most frequently encountered, including resistance to change, failure to deliver software frequently, siloing of work, lack of Product Owner empowerment, and the putting of politics before process. Read more at Coveros.
To Lead a Digital Transformation, CEOs Must Prioritize. Transformational leadership is about creating an agile organization that can detect what type of change is essential and respond quickly with the most competitive solution, says Laurent-Pierre Baculard in Harvard Business Review. The problem however, is that that these efforts tend to be ad-hoc and uncoordinated. CEOs need a holistic view of the digital threats and opportunities facing key parts of the business, and a way to link them to an overall vision. Read more at Harvard Business Review.
How to get agile to work at your company. The need for clear sponsorship in an agile transformation is highlighted in this article from CIO Magazine. Transformation requires a shift in thinking for all parties involved and the full support of C-suite leadership, the business and developer teams. Much of this shift is cultural, especially when people are more familiar with stage-gated waterfall practices. "Getting everyone aligned with how the new process will work is a challenge", the article observes. "In addition, some people have misconceptions and/or lack of understanding to what agile really means." Read more at CIO Magazine.
Agile in the UK Government - An Insider Reveals All. How much, if anything, have GDS achieved? Are £12 billion IT disasters still likely to happen? Key takeaways from this exposé include:
12 Failure Modes of an Agile Transformation. Among Jean Tabaka's many contributions to the betterment of agile practice was this article, in which she identified “12 Agile Adoption Failure Modes”. "It’s imperative that we look not just at Agile adoption, but at Agile transformation — where organizations move beyond Agile principles within their IT groups to business agility", she said. "To accomplish this, we transform from just doing Agile to being Agile". The failure modes she identified include a lack of executive sponsorship and a failure to transform leader behaviors. Each one of these modes can be viewed as an agile antipattern that inhibits successful agile transformation. Read more on the Rally website.
Scaling Agile at Spotify with Tribes, Squads, Chapters, and Guilds. Spotify, a Swedish startup launched in 2008, now has hundreds of agile developers divided into 30 agile teams spread over 4 cities in 3 timezones. Its approach to achieving agile practice at enterprise scale is widely regarded as a definitive case study in the matter. In this 14 page whitepaper, Henrik Kniberg and Anders Ivarsson explain the Spotify system in terms of team responsibilities, roles, and dependencies. Read more at Tech Crunch, including the implications for Enterprise Lean Startup.
How PayPal rallied a 4,000-strong move to Agile. PayPal’s wholesale move to an Agile methodology was built on ‘four pillars,’ took seven months and changed the way 4,000 IT and product people did their jobs. Read more on TechTarget about how it got done.
How Pairing & Swarming Work & Why They Will Improve Your Products. Swarming is a behavioral pattern that many teams would like to apply, but which they find difficult to implement. In this article Johanna Rothman compares and contrasts swarming with pairing, why both of these techniques work well, and how they ultimately help to improve product value. Read more at The Agile Zone.
Unspoken Agile Topics. Companies often decide to introduce Agile practices without thoroughly thinking through why they are doing it. For such companies, Agile adoption has almost taken the form of a fashion statement, a way to prove to themselves that they are up to date with others. In this paper Gene Gendel of Global Investment Bank looks at the impact of these problems on agile transformation, how it is measured, and at how it is implemented in terms of team roles and best practices. Read more at scrumalliance.org.
Empirically managed agile transformation
Agile Development in Practice: the official reference book
"The development of complex systems is fraught with difficulty. Many organizations have taken the vocabulary and outward form of agile practice, but without sponsoring the deep change that agile adoption genuinely requires. The benefits they hope for escape them...and their projects continue to fail. All too often, nothing really changes at all. This book tells you what agile practice is really about, and how to achieve it. The essential components of an agile way of working are laid out chapter by chapter. The book concludes with a concise treatment of the agile transformation problem, and shows how to leverage agile patterns and practices in order to resolve it."
Agile Development in Practice is available from amazon.com.
Pattern of the Month
Uncommitment (antipattern)
Proverbs: The sluggard who does not sow in season does not reap at the harvest
Motivation:
Team members who do not value the Product or Sprint Goal, or who are
allowed to accept other priorities, may be tempted to admit unplanned
work into a Sprint. This can include work from forceful parties which
the Product Owner does not value. Product Owners who do not value the
Product or Sprint Goal can themselves be tempted to trade unplanned work
into a Sprint Backlog to the detriment of value produced.
Structure:
Team members collaborate with each other in a workstream. The
workstream may encompass either Business As Usual activities or project
work delivered in multiple sprints with sprint goals. However, product
ownership will be weak and any sprint goals may be unclear. Team
membership may be ill-defined and/or the backlog may not be respected by
them.
Applicability: Clear goals and
the careful management of time-boxes are central to agile methods such
as Scrum and DSDM. Business As Usual work can encourage uncommitment in
so far as there may be no time-boxed goal to commit to. As such, Kanban
teams can face unique challenges regarding product ownership, the
replanning of work, and team commitment to any forecasts made.
Consequences:
A Product Owner who does not value the product will have little
incentive to help teams formulate Sprint goals, or to exert pull on a
team for delivery. Team purpose and/or discipline will therefore be
compromised. Sprint Goals may not be met, or the value delivered may be
less than that expected.
Implementation:
Uncommitment can take either of two forms: uncommitment by the team to a
forecast backlog of work, or uncommitment by a product owner to the
product being delivered. The result is the same in either case: the
planned backlog will be compromised and the expected value is unlikely
to be delivered.
See Also: Time Theft, Unbounded Team
Agile Transformation in the News
AUGUST 3 2017: Software created in corporate "innovation centers" often fails to make the break into mainstream development. In this CIO Magazine article, TD Ameritrade CIO Vijay Sankaran explains how he intends to break that cycle. Sankaran says he has turned to agile practices, design thinking and lean startup methodologies for quickly building adaptive products. "There is an opportunity to leverage digital technologies to continue to disrupt" he claims. "If you don't do the devops piece in parallel where you have automated testing and continuous integration and you can deploy packages automatically you create a bottleneck in a different place". Note: Sankaran says this has allowed the innovation center to deliver prototypes in one to three months and minimally viable commercial products in only six months.
JUNE 21 2017: The Department of Homeland Security has canceled a large-scale project to make agile software development services easier for agencies to buy, reports Nextgov.com. The initiative let agencies depart from the old acquisition process which could take months or years to work through bids. However, the new system proved too complex to be viable, with 100 companies participating in half-day technical challenges which had not been adequately prepared for, resulting in a "logistical nightmare". Bidders who didn't make the cut were then in a position to claim that the acquisition process was unfair.
MAY 3 2017: The UK 'wastes billions every year' on failed agile projects, according to this article in IT Pro. It reports that "More than half of CIOs think the methodology has been discredited and is just an IT fad". Reference is made to a survey in which it has been found that "Just under one third (32%) of projects are failing because the teams are too geographically dispersed, while 34% have failed because the teams didn't plan before getting started or didn't plan sufficiently as the project developed". Note: Geographical dispersion is a symptom of the unbounded team antipattern while a failure to prepare suggests uncommitment on the part of stakeholders who may be too busy. Inspection and adaptation underpins an agile way of working, where replanning ought to occur on an ongoing basis.
APRIL 3 2017: The Register reports that "hundreds of millions of pounds" have been wasted on a supposedly agile delivery program for digitizing the UK's judicial court services. "Particular scorn has been directed at the so-called 'agile experts' who have been in charge of managing the programme. 'There is no plan, no artefacts, no direction, just constant excuses,' said one insider. 'How they can still be in place as well as still being allowed to recruit 'experts' with absolutely no delivery after 30 months is scandalous.'" The Register also cites an insider as saying that "if the programme were following proper agile principles, by this stage there ought to be at least a dozen meaningful services available across the criminal justice system". Note: A failure to plan and release value incrementally is contrary to an agile way-of-working, as there can be no empirical evidence of progress. The result in this case has been an escalation of commitment when controlled failure may have been the wiser option.
MARCH 1 2017: Eastern Foundry, a collaborative workplace for Federal Government contractors started by military veterans, has spoken out in defense of the 18F initiative. Writing in Nextgov, they point out that although the 18F initiative has struggled, "there is a cost to opposing innovation and forcing reform to move at the pace of government acquisition...18F was created out of a massive procurement failure, and it has moved mountains to help agencies and the nation avoid such costly errors".
FEBRUARY 17 2017: A report by diginomica asserts that although growth has been modest, large enterprises are nonetheless buying in to the agile development approach. In this article Jon Reed looks at Computer Economics’ new survey on agile development, and gains insight into the continued relevance of agile practice and why barriers to adoption persist. Note: Two essential patterns are elicited as a takeaway: "Agile is really about iterative cycles. When you combine agile with a minimum viable product approach, you have a huge edge building relevant software that customers/users actually feel like using".
JANUARY 26 2017: Mobomo, a Washington DC supplier to the U.S Government, has explained its use of agile techniques in the Federal space. In this interview in Application Developer Magazine, they explain the key roles and benefits and the new disciplines which are needed. Note: Many good patterns are identified here, including Product Ownership, Teamwork, Servant Leadership, and Swarm.
JANUARY 12 2017: A new CA Technologies (NASDAQ:CA) global
study indicates that there are great advantages to be had in using agile and Devops techniques together. Mature adopters "realized
significant increases of up to 52 percent in customer satisfaction and
up to 50 percent in employee productivity", according to this report in BusinessWire.
Note: Those advanced users recognized that agile and DevOps practices have to be used jointly before benefits can accrue. Just as DevOps is about changing organizational culture - and is not simply a matter of automation - so agile delivery requires teamwork that bridges the "development-operations divide" and optimizes the value stream.
DECEMBER 20 2016: Software development and operations teams working for the Federal Government need better collaboration and use of automated testing. Writing in CollabNet, Thomas Hooker says that "Agencies using traditional Waterfall methods for software development will need to strongly consider shifting their strategy to Agile practices in order to scale and streamline the software delivery lifecycle". The driver is new standards for compliance, governance, and security. He says these demand that
"the bar should be set high when it comes to the security and quality
of technology". Note: Many agencies struggle to adopt agile practices and resort to the Waterscrumfall antipattern rather than sponsor the deep and pervasive change which is required.
NOVEMBER 10 2016: The digital recruitment head at the Department for Business, Energy and Industrial Strategy (BEIS), has resigned. In a damning blog post picked up and reported by Government Computing, Ann Kempster criticized "...a lack of appetite in departments for real, meaningful transformation. I’m seeing a lack of effective leadership right from the very top of the civil service...There is a lack of vision, lack of ambition and lack of any sort of a plan anywhere". Note: The agile transformation crisis will continue in Government until appropriate sponsorship and vision is in place for the deep and pervasive change that is clearly needed.
NOVEMBER 3 2016: Bipartisan congressional support for the Modernizing Government Technology Act of 2016 (MGT Act) has improved efforts to modernize federal IT, according to Claire Bailey in InsideTechTalk. Concurring with Federal Computer Week that "modernizing big federal IT systems isn’t really about the technology—it’s about solving user’s problems effectively", she says that the need to actively involve customers in agile practice - and to be prepared to fail early and fast - are important steps. Note: Effective Product Ownership underpins successful customer liaison, especially at scale where many stakeholders can be involved. The Controlled Failure pattern should also be mastered if the Death March antipattern is to be avoided.
JULY 18 2016: The Department of Homeland Security agency has only
partially implemented an agile way of working, a congressional audit has
found. While the Citizenship and Immigration Services department
claimed to have adopted agile software
development techniques for its huge, $3.1 billion technology
transformation from paper to digital, success has been patchy. According
to an article in FedScoop, User Story prioritization has been accomplished reasonably well, but integration of deliverables has been poor. Note: The timely release of fully integrated increments of release quality underpins agile practice. Without this the value stream will be inhibited and release orchestration will be compromised. Note also that the audit itself set an expectation that release planning ought to be complete prior to initiating development, which is itself contrary to the agile principles that are supposedly being adopted.
MAY 12 2016: The BBC has been criticised for thinking that agility means "making it up as you go along". According to an article in The Register, the National Audit Office found significant shortcomings in the organization's project for customising the user experience of the MyBBC portal. “MyBBC is an ‘agile’ project that was designed to define benefits as the project progressed," the NAO is quoted as saying, "but two years into the project it still was not clear what the BBC expected MyBBC to achieve overall”. The report goes on to claim that benefits were only defined late in the day, two years after the project had started. Note: The BBC Trust is quoted as saying that the project "did not define its its expected benefits upfront" because it was "an ‘agile’ project where benefits were to be defined as the project progresse[d].” This is a misunderstanding of agile practice, since a Minimum Viable Product ought to be delivered regularly and incrementally, where each MVP is framed in order to validate or invalidate a clear and testable hypothesis. Failure to do so can encourage Vanity Metrics, an antipattern in which success is not measured objectively, but rather by attributes that show the initiative in the best light.
MARCH 8 2016: There is no linear relationship between the number of validated hypotheses and a team’s subsequent success, according to an article in Harvard Business Review.
Ted Ladd provides thought-provoking evidence that although "the Lean
Startup method works", and that validated learning helps teams perform
"almost three times better in the pitch competition than teams that did
not test any hypotheses", more is not necessarily better. In essence, he
has determined that the relationship between validated learning and
subsequent success is non-linear. "There was no linear relationship
between the number of validated hypotheses and a team’s subsequent
success" he observes. "I also found that teams that conducted both
open-ended conversations and more formalized experiments with customers
actually performed worse in the competition than teams that conducted either one or the other during the early stages of venture design". Note:
The author has cited possible explanations for this such as erosion of
confidence due to rapid changes in direction, or managers running out of
patience for continued testing. Both are plausible. Validated learning may be applied as an enabling constraint, but not as a substitute for having a sponsored vision.
JANUARY 25 2016: CompuWare CEO Chris O’Malley has expressed optimism
that mainframe development can bridge the chasm to agile practice. The
main challenge he sees is changing the culture. “The gravity against
change is huge”, he said. He explained that cultural change is hard
because team members will always come up with a "rational" reason that
their particular area can't change…even if they see the necessity of
change for the rest of the organization. Note: The application of the Transformation pattern requires clear executive sponsorship
so that deep and pervasive change can be effected across an enterprise.
Employees must be left in no doubt that their own behaviours and
working practices are expected to change too.
JANUARY 6 2016: According to Hewlett Packard, just one year after its birth the fledgling Impact Hub Berlin "was a single email away
from closure". The Berlin team had to accept they'd failed to implement
their own model: "We teach all the time that we should prototype and we
didn't even prototype", admitted CEO
Nele Kapretz. The trick they used to recover was to "think like a
scientist and treat every iteration of your idea as a chance to test a
hypothesis". Note: this application of the Controlled Failure pattern saved the business by closing the validated learning loop in a timely manner.
Recommended Reading
The agilepatterns.org Briefing Notes. In each briefing an item of topical interest is brought into focus for executive sponsors and enterprise change agents.
- The DevOps Studio (PDF). An
introduction to the "DevOps Studio", a more agile and potentially
bimodal engagement model for IT and Business. (5 minute read).
- Innovation Wars: Trump, Lean Startup, and Zero to One (PDF). The implications of Donald Trump's Whitehouse win for innovation policy are considered. (8 minute read).
- Agile Scaling Frameworks: An Executive Summary (PDF). The most popular agile scaling frameworks are outlined. (2 minute read).
Spreading Scrum through the Enterprise. A well-rounded look at the need for enterprise agile transformation and at the challenges organizations face when scaling Scrum. “Your competitors in your industry are disrupting the industry, so you are either on the disrupting wave or you will be shattered by the disruption wave.” Read more at SD Times.
How to Make Agile Work in Fast-Growing Startups. Looking to take the Agile plunge with your startup? Read these six practical lessons on how to make the transition easier and worthwhile. By Stefan Wolpers. Read more at The Agile Zone.
Beyond Agile Operations: How to Achieve The Holy Grail of Strategic Agility. Most organizations today are still struggling to master agile practice at an operational level. In this article, Steve Denning explains that, although developing these competencies is important, it is agility at a strategic level which must ultimately be achieved. This involves the disruptive creation of new markets where the agile organization brings value as an innovative network of players. Read more at Forbes Magazine.
The Lean Enterprise: Introduction to Massively Disruptive Innovation. At scale, lean organizations have cross-functional teams, culture in which all teams own the product, and embrace an innovative mindset. Failure is not only expected, but encouraged. Read more at The Agile Zone.
The Lean Startup Security Guide. A well thought-out set of measures and safeguards tailored to the security needs of early-stage startups, and which provides a solid foundation for growth. Read more at leanstartupsecurity.com.
Cultivating Disruptive Innovation in the Enterprise.
A look at how innovation "colonies" (also known as innovation
laboratories) help organizations to develop an enterprise lean startup
capability. Read more at SolutionsIQ.
5 Common Pitfalls of #Agile Transformation in the Government (and how to avoid them). A brief examination of some of the pathologies that are most frequently encountered, including resistance to change, failure to deliver software frequently, siloing of work, lack of Product Owner empowerment, and the putting of politics before process. Read more at Coveros.
To Lead a Digital Transformation, CEOs Must Prioritize. Transformational leadership is about creating an agile organization that can detect what type of change is essential and respond quickly with the most competitive solution, says Laurent-Pierre Baculard in Harvard Business Review. The problem however, is that that these efforts tend to be ad-hoc and uncoordinated. CEOs need a holistic view of the digital threats and opportunities facing key parts of the business, and a way to link them to an overall vision. Read more at Harvard Business Review.
How to get agile to work at your company. The need for clear sponsorship in an agile transformation is highlighted in this article from CIO Magazine. Transformation requires a shift in thinking for all parties involved and the full support of C-suite leadership, the business and developer teams. Much of this shift is cultural, especially when people are more familiar with stage-gated waterfall practices. "Getting everyone aligned with how the new process will work is a challenge", the article observes. "In addition, some people have misconceptions and/or lack of understanding to what agile really means." Read more at CIO Magazine.
Agile in the UK Government - An Insider Reveals All. How much, if anything, have GDS achieved? Are £12 billion IT disasters still likely to happen? Key takeaways from this exposé include:
- GDS are trying to transform the UK government, to become digital-by-default
- GDS philosophy puts users first, with a focus on agile principles and practices
- Continuous delivery is happening in many government agencies, including HMRC
- Lots of UK government code is now open source
- Big progress by GDS, but still lots of big challenges
Read more at InfoQ.
Why Agile Fails in Large Enterprises. According to one survey, 68.6 percent of teams have been
involved in a project that they
knew would fail from the start. Agile projects achieved a 71.5
percent success rate, compared with only 62.8 percent of traditional
methods. In this article, Sanjay Zalavadia looks at some common areas
which impact the adoption of agile development processes in large
enterprises. Read more at InfoQ.
12 Failure Modes of an Agile Transformation. Among Jean Tabaka's many contributions to the betterment of agile practice was this article, in which she identified “12 Agile Adoption Failure Modes”. "It’s imperative that we look not just at Agile adoption, but at Agile transformation — where organizations move beyond Agile principles within their IT groups to business agility", she said. "To accomplish this, we transform from just doing Agile to being Agile". The failure modes she identified include a lack of executive sponsorship and a failure to transform leader behaviors. Each one of these modes can be viewed as an agile antipattern that inhibits successful agile transformation. Read more on the Rally website.
Scaling Agile at Spotify with Tribes, Squads, Chapters, and Guilds. Spotify, a Swedish startup launched in 2008, now has hundreds of agile developers divided into 30 agile teams spread over 4 cities in 3 timezones. Its approach to achieving agile practice at enterprise scale is widely regarded as a definitive case study in the matter. In this 14 page whitepaper, Henrik Kniberg and Anders Ivarsson explain the Spotify system in terms of team responsibilities, roles, and dependencies. Read more at Tech Crunch, including the implications for Enterprise Lean Startup.
How PayPal rallied a 4,000-strong move to Agile. PayPal’s wholesale move to an Agile methodology was built on ‘four pillars,’ took seven months and changed the way 4,000 IT and product people did their jobs. Read more on TechTarget about how it got done.
How Pairing & Swarming Work & Why They Will Improve Your Products. Swarming is a behavioral pattern that many teams would like to apply, but which they find difficult to implement. In this article Johanna Rothman compares and contrasts swarming with pairing, why both of these techniques work well, and how they ultimately help to improve product value. Read more at The Agile Zone.
Unspoken Agile Topics. Companies often decide to introduce Agile practices without thoroughly thinking through why they are doing it. For such companies, Agile adoption has almost taken the form of a fashion statement, a way to prove to themselves that they are up to date with others. In this paper Gene Gendel of Global Investment Bank looks at the impact of these problems on agile transformation, how it is measured, and at how it is implemented in terms of team roles and best practices. Read more at scrumalliance.org.
Why You Should Limit Work In Progress. "Sometimes I sit down at the end of a day or even a week and am dismayed that I don’t seem to have really accomplished anything. I know I was very busy and I know I was working hard on important things, but why don’t I have anything to show for it?" Read more on the importance of Limited WIP in this article by David Hammerslag at BigVisible.
   |
