PiecemealGrowth
Once your organization is primed for change, where do you go from there? How do you start applying the patterns?
The answer is simple: pick a pattern to start with, and then apply the patterns one at a time. We will go into more detail in a few moments, but first a warning: Do not attempt to apply these patterns at once! Do not sit down with the pattern book in one hand, and your organizational plan in the other, and attempt to design your whole organization. These patterns must be applied in a piecemeal fashion. In fact, that is the only way an organization can change effectively; it must grow and mature organically.
There is a style of organizational design that believes in formalism, repeatability, and control. This school of organizational design is typified by ISO 9001 compliance programs as they are usually carried out. Perhaps Osterweil's (now quite out-of-vogue) Process Programming [BibRef-SuttonLernerOsterweil1997] is the epitome of this school of organizational design. Such approaches suggest that if we get everything right up front, all else will follow.
Unfortunately, it is impossible to foresee the complexities that beset even the healthiest organization. Human behavior is one of the most pernicious things to predict because it emerges from thousands of considerations and inputs, each weighted differently, that feed the decision processes behind organizational evolution. This makes it difficult to plan organizational structure. Changing economic conditions and markets, changes in the employment roll, in the law, and even the national mood can upset organizational design.
Some organizational structures change slowly and can be depended on as foundations for the evolution of an organizational design. These structures come not from an analysis of the future, but from an analysis of the past. Such structures can be formalized using techniques such as domain analysis and, in this case, one can make an analogy between domain analysis and patterns. Some of the patterns in this book are like that; more so those in the ProjectManagementPatternLanguage than those in other parts of the book.
But most of the time, successful organizational growth takes place in a piecemeal fashion, in real time. One of the pattern chapters is called PiecemealGrowthPatternLanguage, which contains patterns about how an organization grows and develops — gradually. Note, however, that although you apply the patterns one at a time, they do not operate in isolation from one another. You must consider applying all patterns in a process of piecemeal growth.
In a piecemeal growth environment, the focus is on ongoing repair rather than on forecasting and anticipating. In fact, all design is in some respect an act of ongoing repair; we employ the feedback that comes to our senses from the emerging design to modulate the direction of design from that point on. So much the better that one is dealing with a live system and receiving its feedback than when working in the abstract with just "a design." Nature works the same way. Organizations, and their evolution, seem more to follow the laws of nature than the laws of modular design of design in any field with human-created artifacts. And nature works in the now, with feedback, employing repair.
The piecemeal growth philosophy comes from Alexander's vision of how patterns should be used, and pervades his work. The 6-step process below is derived from his yet unpublished work The Nature of Order. (Volume one has been published so far [BibRef-Alexander2003].) But piecemeal growth also surfaces frequently as a key management strategy. One of the main principles of organizational change in AT&T organizations in the 1980s was that one shouldn't try to change more than three things at once. Culture can change, but it loves stability.
More broadly, the pattern philosophy of piecemeal growth is a broadening of the popular notion (particularly during the late 1980s) of organizational learning. There are several excellent books on organizational learning; our favorite is "Becoming a Learning Organization: Beyond the Learning Curve," by Joop Swieringa and Andre Wierdsma [BibRef-SwieringaWierdsma1992]. There are strong parallels between the organizational learning field and patterns. For example, each believes in building on a small number of principles that generate rich emergent behavior; complex systems of rules don't work [BibRef-SwieringaWierdsma1992, 9].
A pattern-based piecemeal-growth repair process is robust for two major reasons. The first reason is that we don't do random things at random times. The patterns encode wisdom born of experience and follow sequences that have repeatedly worked in the past. Second, the patterns build structures which themselves offer a degree of resiliency under change. DomainExpertiseInRoles and FunctionOwnerAndComponentOwner are good examples of patterns that help an organization ride through common changes. If one organized around expertise related to a given product, the organizational structure would be sensitive to changes in the market. The market can be fickle and tends to change much more rapidly than the expertise associated with a given domain.
FunctionOwnerAndComponentOwner honors the tradition of giving focus to the marketable item; after all, that's where the money is. At the same time it guards the long-term stable structure of the system, its underlying knowledge, and the organization that sustains it by also according ownership on the basis of components. DomainExpertiseInRoles helps the organization build foundations around core competencies rather than around current market (or management) fads. The organizational patterns encode that robustness and experience.
The ProjectManagementPatternLanguage can be an inspiration for principles and structures to get you started. It offers many long-term stable domain structures having to do with organizational structure, certainly for software development. Rough forms of these patterns will fall in place early in the formation of a new organization, and these patterns can be honed, fine-tuned and polished over months or maybe years to make them really shine. While DevelopmentEpisode can be an almost methodological construct, one that can be implemented almost overnight, patterns like WorkFlowsInward have more emergent results that come about over time. And even the seemingly simpler patterns like DevelopmentEpisode are cultural changes that will breed discomfort and take some getting used to. Each pattern is a small foreign element—an upsetting event—in its own right (see DissonancePrecedesResolution). Still, the ProjectManagementPatternLanguage is a good "starter set" for most organizations. But your mileage will vary, and you should defer to your instinct and insight.
That foundation in place, you can slowly make improvements by applying one, or maybe two, patterns at a time. Fundamental to the nature of patterns is that each can be applied in its own right without undue consideration of other patterns. Each pattern encapsulates a set of forces, or trade-offs (see WhatArePatternLanguages? above) that are as independent as possible from the forces in other patterns. Ideally, each can be applied in isolation. Ideally, there is no backtracking.
There are of course limits to this idealism. An organization is a system and a system view can be important to keep you from being blindsided. There is no formula or recipe for combining pattern application with insight; it is indeed your deeper insight that will tell you what the proper mix is. We as authors of this book trust you as shepherd of your organization to build on that insight. We provide some hints in the form of patterns, some insights on how the world tends to work well, and trust that those will serve as inspiration for you. This book is not a medicine cabinet. Patterns are not magic remedies to cure ills (more on this in OrganizationalPatternsAreInspirationRatherThanPrescription).
The Fundamental Process
There is a rhythm to the application of patterns and people tend to underestimate the process that makes patterns work. That process is a process of piecemeal growth. One follows a sequence through the pattern language to increase wholeness, one pattern at a time. How, basically, does this work? Here is a synopsis of the process:
Consider your organization as a whole; get a feeling for how the entire enterprise (development group, department, etc.) is working. Get a feel for its "weak spots." Maybe you just applied another pattern. That left you in a new context.What forces are unresolved in that context, either from incompleteness in the pattern you just applied, or from other forces in the system that have now become visible or of higher priority?
Focus on what can be done to increase wholeness. "Wholeness" here reflects your personal and corporate values. Are you striving for profitability? Then what are the weaknesses in your organization related to profitability? And is profitability really what's killing you right now, or is morale affecting productivity which is in turn affecting profitability? Dig deeper. Reading through the patterns — and particularly their forces — can help you sort this out. Reflection is key: it is important to focus on recurring issues and to avoid reacting to immediate concerns or ones that bear a high priority in the moment.
Find a place where the application of a new pattern — the creation of a new role, the addition of a new group, the restructuring of things — will achieve that goal. Will any of the patterns help that? Do you know of other techniques that will help that — whether they bear the pattern banner or not? (We don't get paid by how many patterns we sell; our satisfaction comes from helping organizations succeed!) Don't get stuck in pattern tunnel vision. But if you do this, take good notes — today's odd heuristic might be tomorrow's pattern, and we want you to write it down.
Apply that pattern or technique locally: think locally, act locally [BibRef-Gabriel2000]. But do it in a way that might also increase the wholeness at the next level in the organizational structure, or in the next larger context or scope.
Strive for balance. Most of the patterns here are communication patterns. Communication is rarely a one-party phenomenon; it affects at least two loci. That means that when you apply most of these patterns there will be some kind of local symmetry. Be attentive to the symmetry and make sure both sides of the communication, structure, or other facet of the pattern are attended to.
Reflect — does it feel right? Does it work? Filter the feedback from the organization; people will generally resist change, but are structure and behavior trending in a direction that increases the wholeness? If not, back out. It is much better to back out at this local level, with respect to the application of a single pattern, than it is to forge blindly ahead and do more damage. Eventually, a good pattern will lead you to a new context — and a new set of forces to balance, and problems to which you can attend.
This process iterates after the pattern has had time to settle in and gain acceptance in the culture. That might take days; it might take months. Again, use your judgement.
Piecemeal growth is guided by sequences. A pattern language is a graph, and there are almost innumerable useful paths through it. There are two cues you can use to know which pattern to apply next. One is to look at the structure of the pattern language. The individual patterns tell about what patterns should come next as refinements and progressive steps, and the numerous figures of the pattern language graphs can also be a guide. The other guides that are useful are the sequences that other organizations have followed. The section on CaseStudies looks at the paths followed by several organizations on their rough road to success. They are stories. Stories can offer powerful insights into business choices, and we offer them to you for that reason. Read them through and look for things that hit home.
When do I apply these patterns?
If you are familiar with Design Patterns in software architecture, you probably feel it is obvious when to apply a pattern: when a problem arises during design or implementation. When do you apply organizational patterns?
Remember that the system you are building is more that of a human organization than a software artifact. Change upsets organizations. Good change comes from a process of consensus, and it takes time and focus to develop consensus. Project retrospectives [BibRef-Kerth2001] are an opportune time to consider the new application of organizational patterns to the organization. Retrospectives are an opportunity to sit back and consider the organization as a whole, to find the pressure points of change that will give rise to the right kinds of emergent structure and behavior. Making changes during a consciously planned retrospective helps avoid making decisions in the heat of battle; it can help lead the team to changes that are more systemic and less reactionary in nature.
It is better to introduce organizational patterns between development cycles than in the middle of a cycle; this may coincide with a project delivery, or a change in technology, or perhaps an externally imposed change in organizational structure. Remember to make changes piecemeal and local.
The patterns in the chapter PeopleAndCodePatternLanguage might be applied as problems arise during development cycles; they are a hybrid between organizational patterns and software design patterns.
Writing your own patterns
Each organization has its own patterns of effective communication and development. There are many different kinds of software development organizations: an organization that develops embedded software is likely to be quite different than one that develops in-house interactive tools. Each of these organizations is typified by its patterns.
The patterns in this book are neither universal individually, nor complete as a set. Each organizational can augment this book's patterns with their own patterns. Again, retrospectives provide an opportunity to capture good patterns and add them to the repertoire of good organizational practices.
See the story of the resurrection of the ParcPlace Systems team in [BibRef-Gabriel1996] for a story of a team that rebuilt itself around a pattern-writing effort.
Master Planning And The Theory Of Constraints
One contemporary management fad is Goldratt's Critical Chain and Theory of Constraints [BibRef-Goldratt1999] which has some similarities to the process mentioned above, but which also has some stark differences. It is similar in that the focus at any given time is local, at a particular juncture of problems. But the pattern approach differs from Goldratt's approach in that there is a broader theory and structure guiding the process, a structure based not only on action/reaction but on encoded experience. Goldratt's techniques are more applicable to industrial and inventory processes that are less tainted by human emotion and dynamics. The pattern process is more suitable to organizations, which have a life of their own.
The greatest danger is to try to take control and to foresee exactly how patterns will work together, and to plan now in a way that anticipates how you will plan in six months' time. The world is more complex than that (see PeopleAreLessPredictableThanCode, below). An organization is an ever-evolving structure and organizational process improvement happens in the now. It pays to know history, and one shouldn't ignore the market and technological trends coming over the horizon, but to plan structure based on predicting human behavior is usually a mistake. This is a difficult thing for most managers: it requires a "letting go." Let go; think locally and act locally; trust your instinct and experience and the experience encoded in the sequences of the pattern language for the rest.
Communication and Organizational Learning
We mentioned above that organizational learning is a key property of effective organizations. Many of the patterns in this book concern effective communication in an organization. But communication can't have long-term impact unless there is group learning. That takes introspection. So the second main component of long-term viable teams is that they take time to introspect. That, of course, builds on good communication skills — and leads to UnityOfPurpose, which is one of the core properties of any effective team.