25
Broaden the Decision Frame to Make Effective Decisions
RICHARD P. LARRICK
No decision process can guarantee a perfect outcome. Humans are not omniscient. There is some uncertainty in both what we know and in what can be controlled. For example, as a company contemplates a set of project investments, organizational decision makers are making a number of judgments about both the nature of the projects - the quality of the personnel involved, the size of an untapped market, the reliability of a new product design - and the nature of the future - changes in demand or changes in the economy over time.
In light of this uncertainty, what characterizes an effective decision? This chapter will argue that a major barrier to good decisions is overly narrow views of a decision, or narrow frames, which pretend that knowledge is complete. The principle for effectively dealing with narrow representations is to follow a process that broadens the decision frame. A broad decision frame takes into account (1) multiple objectives - not just the most salient one at the moment; (2) multiple alternatives - not just the first option that lands on the table; and (3) multiple outcomes that could arise in the near and long term - not just the expected state of the world.
Both the psychological and organizational literatures have identified many tendencies that produce narrow decision representations (Bazerman and Moore, 2009; Miller, 2008). Over the decades, these processes have gone by a variety of technical names that include “functional fixedness,” “satisficing,” “selective perception,” “concreteness,” “anchoring,” “availability,” “confirmation bias,” “predecisional distortion,” “framing,” “accessibility,” and “focalism.” Recent theoretical work on decision making has placed these tendencies in a more general framework (Kahneman, 2003; Stanovich, 1999; see also Arkes, 1991; Kahneman and Lovallo, 1993), arguing that the mind is wired with many “System 1” processes that are intuitive, fast, and automatic, with more deliberate and conscious processes (“System 2”) attempting to monitor and correct these intuitive processes. I will not review all of these intuitive, automatic tendencies but will instead focus on their consequences for decision representations - overly narrow views of objectives, alternatives, and future states of the world. I will then consider a set of tools for broadening the decision frame, including formal tools of decision making, but also many informal practices that range from organizational rules of thumb (“the Five Whys”) to tapping the “Wisdom of Crowds.” I will consider limitations and exceptions to broadening the decision frame before closing with a few illustrative cases of narrow and broad frames.
In 1772, Benjamin Franklin gave the following advice to Joseph Priestley on how to make a difficult decision he was facing:In the affair of so much importance to you, wherein you ask my advice, I cannot, for want of sufficient premises, advise you what to determine, but if you please I will tell you how. When these difficult cases occur, they are difficult, chiefly because while we have them under consideration, all the reasons pro and con are not present to mind at the same time; but sometimes one set present themselves, and at other times another, the first being out of sight. Hence the various purposes or inclinations that alternatively prevail, and the uncertainty that perplexes us. To get over this, my way is to divide half a sheet of paper into two columns; writing over the one Pro, and over the other Con. Then, during three or four days consideration, I put down under the different heads short hints of the different motives, that at different times occur to me, for or against the measure. When I have thus got them all together in one view, I endeavor to estimate their respective weights; and where I find two, one on each side, that seem equal, I strike out the two . . . And, though the weights or reasons cannot be taken with the precision of algebraic quantities . . . I have found great advantage from this kind of equation, in what may be called moral or prudential algebra. (Isaacson, 2003, p. 236)
There are two interesting insights in this passage. First, it sketches a formal if simple decision process that has been frequently celebrated as an early model of more sophisticated decision techniques (Dawes and Corrigan, 1974; Hammond, Keeney, and Raiffa, 1997). It lays out a rough version of what would now be called “cost-benefit analysis.” The 20th century has witnessed the birth and maturation of many heirs to Franklin’s simple technique across a range of fields, including economics, statistics, and business disciplines such as operations research. Over the past few decades, each era’s most popular organizational “best practice” (Total Quality Management, Six Sigma) has tried to spread formal decision frameworks to a broad organizational base. In this chapter, I will give some examples of formal decision techniques to illustrate the principle of “broadening the decision frame” without attempting to be exhaustive (or exhausting).
A second, more subtle insight in Franklin’s passage is his explanation for why he conducts his “prudential algebra” over “three or four days.” In his words, decisions are difficult because “all the reasons pro and con are not present to mind at the same time; but sometimes one set present themselves, and at other times another, the first being out of sight. Hence the various purposes or inclinations that alternatively prevail, and the uncertainty that perplexes us.” A key psychological insight in this passage is at the heart of why decisions makers need broad decision frames: the initial representation of a decision is often incomplete. In Franklin’s view, all the relevant motives and reasons are not immediately salient, but come into view over time. Recent cognitive psychology on memory (Anderson and Spellman, 1995) and inference (Sanbonmatsu, Posavac, Kardes, and Mantel, 1998) suggest the source of the problem: decision makers get “stuck” in one way of thinking about a situation and cannot easily generate and entertain new thoughts. Yet, even as their thinking gets stuck, decision makers are confident that their understanding is complete and their resulting judgments are accurate (Soll and Klayman, 2004). Three illustrations of “narrow frames” follow. This section closes with a description of how the social environment will often reinforce narrow individual frames.
Narrow view of the future
When people are asked to make predictions about future events, they typically have an overly narrow view of what could happen. For example, imagine a company that is weighing a series of investments that each hinge on future inflation. The company relies on internal experts to provide a forecast. One form of narrowness occurs when decision makers accept a point estimate - a single “best guess” number - and act on that number. Point estimates fail to capture the range of outcomes that could result. But a second manifestation of narrowness can occur even when forecasters try to give a range of plausible outcomes. The resulting picture of the future is too narrow. Decades of laboratory research on overconfidence (see Buehler, Griffin, and Ross, 2002 and Soll and Klayman, 2004 for reviews) has shown that people overestimate the predictability of the future, and these results have held up well in field settings. For example, recent research has shown that when corporate financial experts are asked to give a range of values that has an 80% chance of capturing some future economic variable, the experts give a range that captures the true outcome less than 40% of the time (Ben-David, Harvey, and Graham, 2007). The “truth” is a regular surprise.
Although people make a good faith attempt to use evidence to form an accurate expectation about the future, they focus too much on evidence that is consistent with a single scenario (often an extension of the status quo), and fail to imagine the wider range of possible scenarios that could happen. This is potentially costly to the extent that a wider, more accurate range would dictate pursuing a different, better course of action. For example, an unexpectedly high or low inflation rate might change the valuation of different project investments. Moreover, anticipating the full range of possible future outcomes can lead decision makers to develop a contingency plan that can best meet the opportunities and challenges of different circumstances.
Narrow set of objectives
People often pursue a narrow list of objectives in their decisions. For example, in hiring new employees, managers will typically focus on the goals of finding someone with technical ability, appropriate training, and at the right salary, as they should. But a variety of other objectives might also matter to the hiring decision, such as a new hire’s leadership potential, ability to work in teams, or ability to shift to new tasks later. Similarly, in evaluating new product lines, senior-level managers will no doubt focus on profitability; however, they might also wish to consider less obvious objectives, such as diversification of product lines, development of new organizational capabilities, and development opportunities for personnel.
Where does the process of weighing objectives go wrong? One problem is that individuals evaluate choice options without considering their objectives at all (effectively “shooting from the hip”). Even when decision makers generate specific objectives, however, the list is typically incomplete. Often the most salient or easily measured objectives block consideration of other objectives.
In a recent article, Bond, Carlson, and Keeney (2008) provided a compelling empirical demonstration that people generate a narrow set of objectives. In one study, Bond et al. had MBA students generate objectives for a highly self-relevant decision: the things they cared about in choosing an internship. They found that students listed about seven objectives on average and felt satisfied that they had covered everything important. However, when they were then presented with both their own objectives along with a list of objectives generated by others, the students tended to find another seven objectives that were as important as the ones they had generated on their own. Left to their own devices, students had spontaneously generated about half of the objectives that they themselves acknowledged to be most relevant for one of the most important decisions of their life.
In business decisions, this narrow search for objectives is compounded by other organizational and social factors (Cronin and Weingart, 2007). Differences in training, experience, and function lead individuals to focus on only a subset of the objectives that are relevant to a decision from an organizational perspective (e.g. product costs, development time, materials, features, service support, etc.). Organizational mechanisms, such as cross-functional teams or job rotations, are designed precisely to ensure that decisions are made with a broader set of objectives.
Narrow set of alternatives
As a final example, people often consider an overly narrow set of choice alternatives. The alternatives might be narrow in number (e.g. considering only one job candidate) or narrow in scope (e.g. considering only job candidates with an accounting degree). One reason for a narrow consideration of choice options is due to organizational dynamics: a new hire or a new project may be suggested at a meeting without any prior consideration of objectives or options, and is then considered in isolation.
A more subtle form of narrowness, however, arises from individual cognitive processes. Research on creativity has shown that once one approach to solving a problem is generated, it is hard to generate new approaches (Chrysikou and Weisberg, 2005). Similarly, research in marketing has shown that thinking of one brand for a consumer good can actually block thinking of other brands (Alba and Chattopadhyay, 1986). By extension, organizational decisions that involve generating a range of options (e.g. job candidates, solutions to a manufacturing problem, etc.) are likely to suffer from the same “functional fixedness” once initial alternatives are generated. However, just as with objectives, decision makers do not recognize their limited ability to generate alternatives. Research has found that decision makers report being satisfied that they have generated a broad and complete set of choice alternatives even when they fail to generate many relevant options (Gettys, Pliske, Manning, and Casey, 1987).
The compounding influence of the social environment
In all of the foregoing examples, the basic cause of narrow framing in individuals is cognitive: associative memory processes lead people to start with a set of assumptions and then recruit evidence in a way that is consistent with the initial view. There is no compensating psychological tendency to spontaneously search broadly for a differing mix of objectives, alternatives, and scenarios. In part, failure to search further is a product of minimizing effort; but the more subtle obstacle is that the information that comes to mind seems complete and coherent, reducing the feeling of uncertainty that would prompt further search.
These individual cognitive tendencies are only part of the story. These individual factors then tend to be reinforced through social and organizational processes. Common training, common experiences, and frequent interaction all have the effect of leading to shared views on problems (Cronin and Weingart, 2007). Although shared views can be beneficial because they facilitate communication and coordination of efforts, they can be harmful when they perpetuate narrow perspectives on a decision problem. For example, marketers who are trained at the same school and work at the same firm on the same product at the same time will undoubtedly develop expertise, but it is an expertise that will be largely overlapping. Through training, experience, and discussion, they will tend to think about problems in the same way - which product features are best, which markets are most promising, which form of media to emphasize, which analysis to follow, etc. To the degree that the group members possess a particular insight, that insight will be shared. However, to the degree that the group members possess blindspots in their thinking, they too will be shared. Ironically, a like-minded group is a poor source for new objectives, alternatives, and future scenarios, but more confident in its ability - the consistency of perspective across colleagues leads each individual to feel validated and confident in his or her view of decisions.
Network sociologists have made similar arguments about how social structure limits diversity of perspective. They argue that dense network structures - in which clusters of people frequently interact - lead to overlapping individual perspectives on problems (Burt, 1992). Empirical research has shown that decision makers who depend on dense networks alone perform more poorly than those who are connected to non-redundant knowledge from outside that dense network (see Chapter 16).
Other organizational and social processes reinforce narrow frames because they discourage independence and dissent. A group that starts a brainstorming session without first asking each member to generate his or her own views on a problem risks having a specific view emerge early in the discussion that then frames everyone’s view of the problem. Finally, even when there are divergent, independent perspectives, they may not be expressed if the group environment focuses on harmony - or conformity to a boss’s opinion - over debate (see Chapters 17 and 18). In sum, although narrow representations may occur at the individual level for basic cognitive reasons, they are often compounded by the social environment. People surrounded by like-minded peers, who follow poor group processes, or who are located in dense networks will tend to share the same narrow frame.
Formal techniques
This section reviews three representative formal techniques for broadening the frame. Although not exhaustive, the three techniques have been chosen for their applicability to many common decisions.
Considering multiple attributes and alternatives. Choosing among alternatives is typically difficult because there is no alternative that is superior on all dimensions. Instead, one has to forego an alternative that may be superior on a less important dimension to gain superiority on more important dimensions. Multiattribute choice techniques prescribe that you think through a wide range of objectives you care about and the attributes that would predict achieving those objectives.
To make this problem concrete, imagine a prototypical organizational hiring process. A set of candidates has applied and your unit has information on each candidate (degree, past work experience, letters of recommendation, and so on). You also have a set of objectives in mind in the hiring process: you need to hire someone who has strong accounting training but who can also manage a team and can, with time, move into a leadership role. The information you have on each candidate defines the attribute levels for a given candidate (accounting degree versus marketing degree; five years of managing others versus three years of managing others). These attribute levels help you evaluate how well each candidate can meet an objective. If a person’s attribute level for accounting experience consists entirely of “prior experience with spreadsheets,” that person does not fare well on an important attribute.
With this information in hand, you can evaluate each candidate on the level they demonstrate for each attribute. With luck, someone has strong levels on all attributes. More likely, however, one alternative will be strong on some but not all attributes. The decision maker must then make tradeoffs and decide whether accounting skills are more or less important than team skills. This last step is essentially a “weighting” step - how important are the differences between candidates on each attribute to achieving your objectives? It should be noted that attributes need not be quantitative or objective - if there is an important subjective dimension, such as collegiality, this can be assessed and weighed in the decision.
This very general form of a choice process, anticipated by Franklin’s quote, is the raw structure for a host of decision-making techniques. The basic structure is to break a problem down into alternatives and attributes and think about the importance of the attributes. The actual application of the method can take a variety of forms. When there is a large set of data that allows one to connect the attributes of past alternatives to subsequent success and failure, one can statistically regress outcomes on attributes to see which attributes are important in predicting success. The attribute weights in this case are derived statistically and can be applied in subsequent decisions - with a few caveats. The method does assume that the predictive value of attributes is reasonably stable and that one is predicting to other cases “inside the range” of what has been observed in the past. Various crises in the 1990s and 2000s, such as the failure at Long Term Capital Management and the subprime mortgage credit crisis, have illustrated problems of predicting “outside the range” of what was observed in the past (often compounded by other problems created by misaligned incentives and insufficient monitoring).
When there are no past data, there are decision analytic techniques for putting weights on attributes (see Clemen and Reilly, 2004; Hammond, Keeney, and Raiffa, 1997, for a user-friendly starting point). Some of these processes are “top down” - one consciously weighs the importance of tradeoffs across attributes - and some are “bottom up” - one simply ranks a set of alternatives that differ across attributes and captures the implicit weights one is giving to each attribute (this technique is known as “conjoint analysis” in marketing). It can be quite useful to use both steps to see whether the head (“top down”) and gut (“bottom up”) agree. If they do not agree, it can help identify a missing objective or force one to reconsider weights on attributes.
Once a multiattribute analysis is done, it can help inform subsequent decisions. And, in some cases, it can serve as a formula for future decisions that could replace individual discretion. When would this be desirable? First, a good deal of research in psychology has shown that when decision makers make a series of choices intuitively - without an explicit formula - decision makers make less consistent and accurate choices. By looking at alternatives case by case, decision makers inconsistently apply weights across cases (Arkes, 2003) and over-react to fleeting bits of information about specific alternatives (echoing Franklin’s insight about the vagaries of “what comes to mind”). Second, basic issues of fairness can actually support the use of formulas over individual choice. For example, when mortgage decisions are left to individual loan officers, various prejudices can influence their judgments; even if they have no prejudices, officers can still create procedurally unfair outcomes if they weight the same attributes differently for different potential customers. These inconsistencies may be even more damaging in promotion and raise decisions inside an organization, since the shadow of the future is longer between employee and company, and friends share private information. A formula can be an organizational tool for ensuring neutrality and consistency in decisions (Arkes, 2003).
The multiattribute choice process broadens the decision frame by making explicit the need to consider multiple objectives, multiple attributes that help you achieve the objectives, and multiple alternatives. It directly avoids the trap of paying attention to only one objective or alternative. By making tradeoffs salient, it prevents decision makers from being surprised later when unattractive aspects of an alternative become apparent (“this person is good with numbers but lousy with people - what do we do now?”). By paying attention to all attributes, good and bad, early in the decision process, steps can be taken to mitigate weaknesses. It may be easier to give someone relevant management experience on the job than to teach them accounting (or vice versa), and a plan can be made accordingly. Finally, by calling attention to the good and bad early on, it can lead decision makers to be more creative - to acknowledge that there is no superior alternative at this point, and to explore in new ways (or with more investment) for an alternative that is strong on all relevant attributes. A clear understanding of multiple objectives creates a necessity that becomes the mother of invention.
Assessing and weighing future states of the world. A second formal technique for broadening the frame is thinking through the range and likelihood of different uncontrollable events that could influence the outcome of a decision. A course of action that is attractive under one state of the world (e.g. demand for our product stays strong) could be disastrous under another state of the world (e.g. demand declines). The first step in assessing risk is to generate the range of possible scenarios that could arise in the future that influence achieving your objectives. With such an understanding, one can then assess both the likelihood of each scenario and how it would affect the outcomes attached to each alternative under consideration. Finally, one can generate new, contingent ways of responding to each scenario (Schoemaker, 2004).
To continue the hiring decision example, a company may be facing an increase in demand for a product at the moment, but needs to anticipate the need for work over various periods of time (a quarter, a year, etc.). Decisions about hiring for a current increase in demand depend on how controllable decisions (e.g. opening a new market) and uncontrollable factors (e.g. changes in demand) affect long-term needs. Uncontrollable factors, such as a change in demand, require that the company assess the range of plausible changes in demand and attach probabilities to each change. This picture of the uncertain future can then be used to calculate the “expected value” of various hiring options for dealing with a current increase in demand, such as paying overtime to current employees, hiring temps, or hiring new permanent employees. Flexibility in reducing staff is valuable if there is a significant risk of a decline in demand over the long term; adding well-trained, permanent employees is more valuable if there is a strong likelihood of continued demand.
Risk assessment broadens the decision frame by forcing the decision maker to imagine a range of scenarios, think through their likelihoods, and assess their impact on the value of different choice alternatives. It overcomes a basic habit of planning for one future state of the world. Often the envisioned future is a rosy one (Buehler et al., 2002), which is especially likely for people who are pushing a specific plan or course of action, but it can also be a gloomy one that leads decision makers to be unprepared for unexpected opportunities. Having assessed scenarios, likelihoods, and outcomes, one can then formally calculate “expected values” that weight outcomes by their likelihoods. One can also assess whether an alternative could lead to a wide range of good and bad outcomes and decide whether one would prefer a safer alternative that has a smaller range of highs and lows (technically, an alternative that has lower variance). Most importantly from a decision-making perspective, anticipating future scenarios allows one to make contingent plans - to create alternatives that have known decision points at which a planned new course of action would be pursued.
Using “all four cells” to make accurate predictions. The final formal tool is the simplest: do the right checks to see whether an attribute predicts an outcome. Consider one more example related to predicting job performance. Many years ago, Management Focus Magazine reported a research study claiming that pet ownership as a child predicted future success as a leader. How did the researchers reach this conclusion? They surveyed 74 Fortune 500 CEOs and found that 94% had owned a dog, cat, or both as a child. In interviews, the CEOs observed that “pet ownership had helped them to develop many of the positive character traits that make good managers today, including responsibility, empathy, respect for other living beings, generosity, and good communication skills.” (One is tempted to ask whether “communication skills” include beg, sit, and heel. . . .) Should pet ownership be an attribute in hiring decisions?
A little reflection reveals two shortcomings with this study. First, the study is looking at only two cells in a four cell table. Specifically, it is examining the presence or absence of pet ownership among CEOs. The remaining two cells of interest are the presence or absence of pet ownership among non-CEOs. If most people have a dog or cat at some point in their childhood, then we have not learned much from the 94% figure among CEOs. This is a classic problem of sampling on the dependent variable, and there are many famous examples. For example, the research underlying In Search of Excellence in the 1980s looked at a sample of successful firms to uncover their common practices, such as “management by walking around.” One danger with this “two cell” method is that it invites imbuing ordinary activities with special qualities. Until all four cells are examined, one cannot even tell if there is a relationship between the attribute and the outcome. (For example, managers may also be walking around unsuccessful companies, but there it is called “micromanaging.”) In the extreme, we can observe that all CEOs brush their teeth and all successful companies have buildings, but we have not learned anything about predicting success. (See also Denrell, 2003, for how managers draw poor conclusions from “survivors” in risky environments.)
The second shortcoming of the pet study is that, even if CEOs show a higher rate of childhood pet ownership, this may not be the causal factor in their success (despite enhanced communication skills, etc.). Families that have other characteristics that lead to success in business - money, educational opportunities, job opportunities, and social networks - may be more likely to own pets. This is a classic problem of interpreting correlation as causation.
The business press and our day-to-day decisions often rely on casual observation that “sees” a relationship based on only two cells. GE seems to produce a large number of managers who go on to run other companies. Should other companies copy their system for training and promoting leaders? It is also the case that GE produces a staggeringly large number of non-CEOs as well. The key question in evaluating their ability to train and promote is whether they produce at a higher rate given their employee base. In using attributes in a decision, one must broaden the decision frame by checking all four cells in a covariation table to see if there is a relationship, and then, if there is, asking what additional factors might be serving as causes of the relationship.
Barriers to adopting quantitative techniques. Highly quantitative versions of the techniques described above are, in theory, already present in many organizations. These techniques have been taught in business and engineering schools for years. They have also been part of the major management movements of the past two decades. For example, both Total Quality Management (TQM) and Six Sigma have major components that focus on risk assessment, causal analysis, and processes for weighing attributes and alternatives.
However, the everyday organizational reaction to using formal, quantitative techniques is quite mixed (Zbaracki, 1998). Employees are cynical when such techniques are proposed in new programs because they are often introduced by outsiders or from above. The mathematical nature of the techniques appeals to some employees, but requires extensive training for many employees. Once learned, the techniques are time consuming to use. And, finally, these formal techniques face a classic problem described in the diffusion of innovation literature: the benefits of adoption are not easily demonstrated. Thus there is a tendency for the formal, quantitative aspects of these practices to wither in the process of adoption (Zbaracki, 1998).
The major benefits of the practices described above do not depend on rigorous quantification but on attempting to follow the general process. The more important factor in broadening the frame is to strive for a complete representation of the decision problem. Breadth is accomplished by thinking about attributes, alternatives, tradeoffs, and scenarios. Because extensive quantification in decision making can be offputting to many organizational decision makers, I next consider informal techniques that also preserve the benefits of broadening the frame with even less formality.
Informal techniques
Heath, Larrick, and Klayman (1998) have proposed that many of the most effective decision techniques that are actually used in organizations are less formal than the abstract, quantitative techniques described in economics and statistics. Instead, these informal decision techniques, which they called “cognitive repairs,” fix narrow frames by being simple, specific, home-grown, and social. This section briefly describes several of these informal practices.
Informal techniques for broadening the search for information. Some simple techniques force decision makers to search for more information without directing decision makers to specific information. For example, one basic technique that has proven to help a variety of decision shortcomings is to “consider an alternative” to the current conclusion (Hirt and Markman, 1995). The technique simply requires a person to ask “why might my favorite answer be wrong?” This question can be applied to a range of decision problems. If a future scenario seems likely, one can ask why it might not happen. If a single best alternative has been proposed, one can ask how other options might be better. If pets seem to predict being a CEO, one can ask whether they also predict not being a CEO. This is one of the few informal techniques that has proven to have broad utility (summarized in Larrick, 2004; see also Herzog and Hertwig, 2009; Kray and Galinsky, 2003).
A second simple technique for broadening search is the “Five Whys” which has been a regular component of TQM and Six Sigma. The process is simply one of starting with a problem, asking why, and then following up each answer with an additional why. The goal is to go beyond a superficial, narrow understanding of a situation to a deeper understanding. For example, the Five Whys can be used to identify a decision maker’s basic objectives. In a hiring decision, one can ask: Why do we need someone with an accounting degree? Because the person needs to have a good understanding of the financials. Why does the person need a good understanding of the financials? Because he or she will have primary responsibility for cutting costs. Why is the emphasis on cutting costs? There has been no revenue growth in recent years. And so on. The result of asking Five Whys will be more fundamental objectives that could be pursued through a wider variety of attributes than initially considered. It can reveal that an accounting degree is not the only way to address the fundamental objectives, allowing one to consider alternatives that are strong on other attributes to be considered. Of course, like any informal technique, the Five Whys is coarse and can go wrong: one can easily ask why owning pets would make one a better CEO and talk oneself into an elaborate (false) theory of communication skills.
Informal techniques for using a broader set of attributes and alternatives in decisions. A second class of informal techniques overcomes narrow frames by requiring decision makers to use checklists for gathering information and evaluating alternatives. For example, one bank’s commercial loan department required that officers evaluate potential customers on the “5 Cs” - Collateral, Capacity, Capital, Conditions, and Character (Heath et al., 1998). Although necessarily imprecise, specifying these attributes helps officers avoid over-reacting to one favorable or unfavorable piece information as they assess a given loan candidate. Similarly, recent research in medicine has shown that what should be habitual decisions for doctors can also be helped by checklists. For example, a simple five-point hygiene checklist helped reduce infection rates at Johns Hopkins hospital from 11% to 0% (Gawande, 2007). Similar protocols have proved useful in diagnosis by encouraging doctors to systematically consider a wider range of symptoms and disease scenarios (Heath et al., 1998). Checklists, of course, are not perfect. They are likely to provoke resistance from users at first, since they seem controlling. They will be more acceptable if they are home-grown or inherited from experts. Moreover, checklists are only as good as the validity of the items on the checklist and run the risk of omitting important considerations. The bet one makes with a checklist is that individuals with narrow frames would omit even more of the important considerations.
Informal techniques for broadening diversity of perspective. A final set of informal techniques takes advantage of the fact that combining multiple narrow frames can create a broad, complete understanding of a decision. Specifically, if the independent perspectives of multiple people are pooled together, the resulting knowledge is broader and more complete than the view of any one person. However, the benefit of drawing on multiple people is highly contingent on tapping a set of people who hold diverse perspectives initially and then following procedures that preserve it.
Consider a technique used in TQM, known as a Kokai watch (Walton, 1990), that involved assembling a team to evaluate a current procedure and look for improvements. The team was deliberately assembled with “non-experts” - people who were managers or employees in another department and not expert in the domain they were studying. The logic of using non-experts is that those wrapped up in a routine cannot step back to question the process; people with fresh eyes, however, can reason by analogy to a broader set of knowledge and experience. In general, decisions that involve diagnosing problems and generating new alternatives can benefit by having non-experts provide fresh eyes. In a similar vein, Motorola would compose cross-functional teams for new product designs to ensure diversity of perspective (on objectives, attributes, alternatives, future scenarios, etc.), but would then break up the team at the end of an assignment (Heath et al., 1998). One could argue that once a team meshes - and especially if it is successful - it should be preserved for the next assignment. Motorola, however, recognized that even crossfunctional teams can come to share a common perspective on decisions that could limit the objectives and alternatives in subsequent assignments. By mixing teams, there was a constant tension created by different perspectives coming into contact.
The value of using groups (see Chapters 15-18) to broaden frames has received increasing recognition in recent years, and was popularized as the “wisdom of crowds” in a book with that title (Surowiecki, 2004). The main factors that make a crowd wise are diversity and independence. Unfortunately, classic brainstorming can undermine independence of thought to the extent that early suggestions can entrain everyone’s subsequent ideas. Thus, the best group techniques preserve initial differences in perspective by having individuals think about a problem alone (this stage is often called a “nominal” group), but then pool the information so that others can react to it. Empirical research shows that nominal groups will generate a much broader set of objectives and alternatives than any individual can (Bond et al., 2008; Gettys et al., 1987). If desired, one can use the frequency of naming specific objectives and alternatives as a “vote” for their importance, with the caveat that truly creative ideas will tend to be rare ones.
Nominal groups are also superior to individuals in estimating unknown, uncertain outcomes. Each person’s glimpse of the unknown is imperfect. However, when a large number of such imperfect estimates are averaged together, extreme errors cancel out, and the resulting group average will be closer to the truth than the answer of the average individual (this holds whenever the extremes in the group bracket the truth), which is a common pattern (Larrick, Mannes, and Soll, 2009). By way of illustration, the Wall Street Journal publishes twice-yearly economic forecasts from 50 professional economists. These data are useful for seeing the range of outcomes they think are possible (providing a sense of possible future scenarios); moreover, the mean estimate tends to be more accurate than the majority of judges in the panel. Although the WSJ routinely celebrates which economist “wins” in a given six month period after the fact, there is no reliable way to pick a single expert a priori that can beat the group average over time (Larrick et al., 2008). Averaging judgments is also helpful in quantifying subjective attributes in a multiattribute choice process: one can ask multiple interviewers to rate different job candidates on collegiality and use their average judgment for each candidate in a decision.
Although brainstorming is the prototype for group techniques, many hybrid group processes are possible with the advent of new forms of electronic communication (Chapter 32). Organizations are finding increasing ways to tap the wisdom of their own employees, customers, and partners in events like innovation jams and prediction markets. These can be efficient (even fun) ways for organizations to aggregate hundreds of narrow individual frames to broaden the decision frames of individual decision makers higher in the organization.
Groups, of course, can talk themselves into poor decisions that they then hold with high confidence because of the perceived widespread support of the conclusion. To prevent conformity pressures and groupthink, it is essential that groups encourage diversity of perspective, independence of thinking, and a willingness to express dissent (Chapters 15 to 18).
This chapter has argued for taking a broad view of objectives, attributes, alternatives, and scenarios in making decisions. This picture of effective decision making is consistent with a large body of theoretical and empirical work. However, there are potential limitations to using the principle. This section considers two possible challenges to the principle of broadening the frame. The first challenge is practical considerations about implementation. The second challenge is recent proposals to rely on quick, automatic, intuitive processes; in short, to “blink” rather than think.
Practical limits on broad frames. There are several practical limitations to broadening the frame:• Elaborate techniques, such as multiattribute choice processes, are time consuming, and risk “analysis paralysis.” (How do you know when you have reviewed enough attributes and alternatives?) As decisions become more complex, decision makers often rely on simplifying processes to make the task manageable (Payne, Bettman, and Johnson, 1993). Recent research has found that too much choice can lead decision makers to avoid making a choice at all. The diminishing marginal benefit of considering more attributes and alternatives needs to be balanced against the potential cost of overwhelming people with information. Traditionally, decision makers suffered from too little information. It may be the case that technology is able to capture so much information that selecting alternatives and attributes - rather than seeking them - is the greater challenge.
• Recommendations that involve soliciting objectives and alternatives from many different parties risk introducing conflict, politics, and negotiation. However, even in a politically charged environment, an awareness of objectives and alternatives can lead to better decisions through negotiation - specifically, good decisions can be made if each party sacrifices unimportant attributes to gain more desirable attributes. And participation at least has one silver lining. Broad participation in generating objectives, alternatives, and scenarios will tend to increase commitment to a final decision, especially if the process of collecting and weighing information is seen as procedurally fair.
• More comprehensive processes may increase confidence and satisfaction with a decision, but, if the processes make obvious stark tradeoffs, they may dampen satisfaction with a final decision.
Blinking rather than thinking. In his book Blink, the New Yorker writer Malcolm Gladwell (2005) popularized the notion that sometimes very quick decisions are highly accurate. This idea has also received serious academic attention in recent years, perhaps as a bit of a backlash to the rational, deliberate models that had been studied in earlier work. Three major academic claims in the spirit of Blink seem to challenge the “broaden the frame” principle.
The first claim is that many decisions can be made better and more quickly by focusing on a single attribute or predictor and then incorporating additional information only if that first piece is inconclusive (a strategy termed “Take the Best,” or TTB, Gigerenzer and Goldstein, 1999). If TTB were generally effective, it would directly undermine the prescription to use a broad frame. Many studies have shown that TTB is effective - specifically, it works well in situations when one attribute is much more important than other attributes, and it works well when the most important attribute is obvious. The problem with these demonstrations, however, is that they are selective. One of the main difficulties in decision making is the uncertainty around identifying the most important attributes and in knowing how much weight to place on them. Empirical research shows that people have a hard time learning which attributes are most important (or predictive). If there is error in identifying the most important attribute, TTB performs much less well than broad strategies that use multiple attributes (Hogarth and Karelaia, 2007; Payne et al., 1993). The case for TTB also has other weaknesses, such as testing it only with attributes that have two levels. In short, using multiple attributes or multiple predictors is a better strategy when information is known with uncertainty.
A second claim that challenges the principle of broaden the frame is that many expert decisions are done quickly and automatically, without extensive weighing of options. Expert decision making has been studied in a number of domains, ranging from tasks such as chess to firefighting. Research by Klein and colleagues (Lipshitz, Klein, Orsaanu, and Salas, 2001) makes a convincing case that experts do process information differently than novices. Specifically, experts have a wide array of experience in their particular task that allows them to quickly recognize a situation and retrieve an effective course of action. The main question for organizational decision making is what conditions support the development of these skills? First, it should be noted that the skills of experts are domain specific - chess masters and firefighters do not develop general decision-making skills that apply to new domains, but only to their own. Second, experts are able to develop skills because they face a very favorable environment for learning - an environment in which a person can experience a broad but recurring set of situations and see the immediate, unambiguous results of his or her actions (Hogarth, 2001). If situations were constantly novel, or feedback were ambiguous or delayed, it would be very difficult to learn how to react effectively - and “experts” in many judgment domains make predictions that are no better than those of novices (Camerer and Johnson, 1991; Tetlock, 2005).
Moreover, it is not quite accurate to say that experts use narrow frames; it is more accurate to describe them as quickly taking in relevant attributes and automatically generating alternatives. These automatic responses are based on broad experience that is quite costly to gain. Research indicates that expertise takes about 10 years of hard, conscious practice to develop (Ericsson, Krampe, and Tesch-Römer1993). Finally, despite the ability to rapidly recognize situations, it is interesting to note that chess experts make a fast initial assessment but then broaden the frame by weighing different alternative actions and anticipating a range of possible scenarios that might follow (Makridakis, Hogarth, and Gaba, 2009).
A third claim that challenges the principle of broaden the frame is the proposal that “unconscious” decision making is superior to “conscious” decision making (Dijksterhuis, Bos, Nordgren, and van Baaren, 2006). In these studies, people were presented with four apartments that differed on 12 attributes, creating 48 pieces of information. Participants saw each piece of information once, in random order; the information was then taken away and decision makers were either told to think about the decision for four minutes or asked to complete a filler task for four minutes. All participants then reported a decision. The filler-task group was more likely than the deliberation group to select the apartment that was superior on more attributes. This paradigm, however, has not stood up well to additional empirical tests. First, deliberating for four minutes on a fairly simple decision is an unusual task; when decision makers are allowed to “self-pace” their decision (which is often relatively quick), they perform as well as the “unconscious” decision makers. Second, when the task is changed so that the magnitude of the differences between alternatives on some attributes is quite large, conscious decision making correctly favors the better option - in this case, an option that has fewer positive attributes but that has a higher expected value (Payne, Samper, Bettman, and Luce, 2008). The Dijksterhuis paradigm also has practical weaknesses. First, it is not a pure instantiation of unconscious decision making - it involved a great deal of conscious consideration of both alternatives and attributes prior to the unconscious decision process. Second, the paradigm does not pass a day-to-day plausibility test - if attribute and alternative information is both available to a decision maker and unfamiliar, why would the person not be able to examine the information as he or she made a decision?
A more compelling argument against deliberation was made in research by Wilson and colleagues (Wilson, Lisle, Schooler, Hodges, Kaaren, and LaFleur, 1993). In this study, student participants listed reasons as they chose a poster for their dorm or apartment; a control group chose a poster without any deliberation. Three weeks later, the group that listed reasons was less happy with their poster than the “gut reaction” group. The authors argue that articulating pros and cons for the poster decision is a poor strategy because the important attributes are hard to articulate (the beauty and style of a poster), leading decision makers to focus on less important but easier to articulate attributes (the color, size, or subject). This is a fair critique of broadening the frame - if broadening the frame brings in irrelevant attributes that distract decision makers from relevant attributes, it will hurt decisions. As I mentioned in the section on multiattribute choice, however, subjective dimensions are fair game for inclusion in the choice process. It would be reasonable when choosing a poster to capture a gut reaction in addition to reasons pro and con and weigh them against each other if they are in conflict. Recent research has started to examine the benefits of combining intuitive and analytic judgments (much like averaging judgments in a crowd) because each type of judgment can compensate for weaknesses in the other.
In popularizing the idea of “blinking,” Gladwell himself does not take a clear stand on when it makes sense to blink and when it makes sense to think. He describes how many intuitive processes, such as stereotyping, are inaccurate and harmful. It is also interesting to note that many of his examples of successful blinking support the principle of broadening the frame. For example, Gladwell reviews John Gottman’s startling research on couples. Gottman is able to watch short videotaped segments of couples interacting early in their relationship and predict whether they will stay together over subsequent years. Moreover, he has identified a handful of specific cues, or attributes, that predict whether couples will stay together (when they have a disagreement, for example, do they show a lack of respect for each other?). As amazing as these results are, however, they are not compelling illustrations of blinking over thinking. First, untrained observers cannot watch the brief clips and make accurate predictions - in this case, fast, automatic, intuitive processing fails. Second, trained observers can make accurate predictions, but they only know what to look for because cues and outcomes were carefully measured for decades in Gottman’s lab. Large regression models were then performed on the data to identify which cues or attributes were most predictive. Regression is the quintessential multiattribute choice technique.
In this section, we consider two cases that illustrate varying degrees of narrow and broad frames. The first case, the Challenger space shuttle analysis, is an infamous example of a narrow frame for checking information. The second case, Seagate, illustrates how individuals often hold narrow objectives, but an organizational process of gathering objectives can broaden the final decision frame.
The Challenger launch decision
After years of glory striving for and reaching the moon, the National Aeronautic and Space Administration (NASA) found itself in the early 1980s slipping from the national limelight. To slip from the limelight meant not only to lose glory, but also to lose stature when requesting funding in each year’s federal budget. NASA had made space travel more economical with the creation of the reusable space shuttle in the late 1970s and had flown over 20 successful missions by 1986. However, orbital flights did not attract the attention of lunar missions and sporadic delays had diluted the interest of NASA’s remaining audience. In the early 1980s, NASA decided to send a civilian into space - a teacher named Krista McAuliffe - to help recapture the imagination of teachers and school children.
Even while NASA was having success with the space shuttle it was discovering problems with the shuttle design. Engineers from the firm that designed the shuttle’s fuel boosters, Morton-Thiokol, had gathered data on various physical problems observed after shuttle flights. In particular, they noted that large rubber gaskets known as O-rings frequently showed wear and signs of “blow-by” (escaped exhaust). These O-rings played a critical role in containing the burning fuel in the shuttle boosters. By the early 1980s, the engineers had suspicions that temperature was related to O-ring problems. (It was later recognized that cold temperatures made the rubber of the O-rings brittle, thereby preventing them from sealing.) As they approached the February 1986 Challenger launch, engineers were concerned that the below-freezing launch temperature was far below that of any previous launch. The engineers raised their concerns with NASA administrators, but they gathered and presented data in a piecemeal fashion which blocked the ability to see any overall picture.
As importantly, all of the data the engineers presented were focused on past problem flights, seven in number, which had occurred at a wide range of temperatures and provided a murky picture. To quantify it, four problem flights had occurred when temperatures were below 65 degrees; three had occurred when temperatures were above 65 degrees. NASA administrators decided that staying on schedule was more important than acting cautiously based on such ambiguous evidence. However, as Tufte (1997) notes in his retrospective analysis, the evidence was not ambiguous - it was incomplete. (And, to be fair, the engineers also did not have access to all the temperature data.) Tufte notes that “the flights without damage provide the statistical leverage necessary to understand the effects of temperature . . . and no single chart contained data on both in relation to each other” (pp. 44-45). In other words, to see all four cells in the Challenger example, one needs to know how many past flights were flown without damage at warm and cold temperatures. The answer: 17 flights had been flown at temperatures above 65 degrees with no problems; no flights were flown at temperatures below 65 degrees without a problem. Using all 24 historical cases, the conditional probability of a problem was 15% above 65 degrees and 100% below 65 degrees. (These results hold for alternative cuts of the data: problems arose on 41% of flights flown at temperatures below 75 degrees (7/17 cases) compared to 0% for flights above 75 degrees (0/7 cases).)
The Challenger incident has become the most famous example of a failure to broaden the frame when assessing a relationship. It shows why it is not enough to look at the relationship between a variable of interest, temperature, and one outcome, failure. One must also look at the relationship between temperature and success. The combined data - all four cells - supported the suspicion of a link between temperature and O-ring failure and made a convincing case for caution when the temperature was below freezing.
Defining objectives at Seagate
In the early 1990s, Seagate Technology was the largest manufacturer of disk drives in the world. It had recently acquired a dozen companies and was working to create a single, integrated company from the disparate parts. Ralph Keeney was approached to help the company identify a comprehensive set of objectives that would help Seagate define its mission (Bond et al., 2008). Keeney started by meeting with each high-level executive separately. In each discussion, he asked the executive to identify “any objectives, hopes, desires, aspirations, or plans (s)he had for the new organization” (p. 65). This open-ended questioning was followed by more tailored questioning that helped respondents consider objectives that he or she touched on only lightly. Keeney then summarized the objectives and sent the summary to each respondent to see if they accurately represented his or her views. Obviously, the future of the firm was an important issue to each of the executives. Executives committed a substantial amount of time to the task and had a chance to reflect on their answers.
By pooling the objectives together, Keeney identified eight overarching objectives (“contribute customer value,” “help employees achieve satisfaction,” and so on) with 39 specific objectives tied to meeting the larger objectives (“increase customer productivity,” “provide opportunities for career growth,” and so on). On average, each individual executive mentioned five of the eight overarching objectives, and 14 of the 39 specific objectives. When the entire set of objectives was organized and presented to senior management, there was a consensus that the aggregate list was what Seagate wanted to achieve. Only one individual had close to half the overall picture.
The Seagate example illustrates how individuals tend to have an incomplete representation of firm objectives. As importantly, it shows one example of a process for broadening the frame - anoint a specific individual to gather information (in this case, a neutral outsider trained in eliciting information). However other processes could substitute for hiring a consultant. A trusted inside member could play the same role. Or electronic mechanisms (see Chapter 32), such as a jam or bulletin board, could be used to gather insights over a period of time. All of these mechanisms will lead to a more complete representation of objectives that should guide both company and, at times, department-level decisions.
CONCLUSION
This chapter has focused on narrow frames as a general problem in decision making. It has not focused on enumerating the many specific biases that have been identified through research, such as honoring sunk costs, forming probability judgments based on what is available in memory, and treating gains and losses differently (interested readers should see Bazerman and Moore, 2009, for a useful review of these specific biases). These specific biases are important and worth knowing about. Most of them, such as “availability,” are examples of how fast, automatic cognitive processes yield an incomplete cognitive representation. The premise of this chapter is that the fundamental problem in decision making is accepting such incomplete representations as complete.
The prescription to decision makers therefore is to use processes that broaden the frame for important decisions. By incorporating more complete information, broad frames will, on average, be more accurate than narrow frames. In the spirit of the chapter’s principle, however, I must acknowledge that future research may identify predictable contingencies when narrow frames - rapid, intuitive, simple representations - are more accurate than broad frames. The bet based on current evidence is on broad frames. Future research will broaden the frame of our understanding.
Alba, J. W., and Chattopadhyay, A. (1986). Salience effects in brand recall. Journal of Marketing Research, 23, 363-369.
Anderson, M. C., and Spellman, B. A. (1995). On the status of inhibitory mechanisms in cognition: Memory retrieval as a model case. Psychological Review, 102, 68-100.
Arkes, H. R. (1991). Costs and benefits of judgment errors: Implications for debiasing. Psychological Bulletin, 110, 486-498.
Arkes, H. R. (2003). The nonuse of psychological research at two federal agencies. Psychological Science, 14, 1-6.
Bazerman, M. H., and Moore, D. A. (2009). Judgment in Managerial Decision Making. Wiley.
Ben-David, I., Harvey, C. R., and Graham, J. R. (2007). Managerial Overconfidence and Corporate Policies. American Financial Association, Chicago, November.
Bond, S. D., Carlson, K. A., and Keeney, R. L. (2008). Generating objectives: Can decision makers articulate what they want? Management Science, 54, 56-70.
Buehler, R., Griffin, D., and Ross, M. (2002). Inside the planning fallacy: The causes and consequences of optimistic time predictions . In T. Gilovich, D. Griffin, and D. Kahneman (eds), Heuristics and Biases: The Psychology of Intuitive Judgment (pp. 250-270). Cambridge: Cambridge University Press.
Burt, R. S. (1992). Structural Holes: The Social Structure of Competition. Cambridge, MA: Harvard University Press.
Camerer, C. F., and Johnson, E. J. (1991). The process-performance paradox in expert judgment: How can experts know so much and predict so badly? In K. A. Ericsson and J. Smith (eds), Towards a General Theory of Expertise: Prospects and Limits (pp. 195-217). New York: Cambridge Press.
Chrysikou, E. G., and Weisberg, R. W. (2005). Following the wrong footsteps: Fixation effects of pictorial examples in a design problem-solving task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 1134-1148.
Clemen, R. T., and Reilly, T. (2004). Making Hard Decisions. Boston: South-Western College Publishing.
Cronin, M. A., and Weingart, L. R. (2007). Representational gaps, information processing, and conflict in functionally diverse teams. Academy of Management Review, 32, 761-773.
Dawes, R. M., and Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95-106.
Denrell, J. (2003). Vicarious learning, under-sampling of failure, and the myths of management. Organization Science, 14, 227-243.
Dijksterhuis, A., Bos, M. W., Nordgren, L. F., and van Baaren, R. B. (2006). On making the right choice: the deliberation-without-attention effect. Science, 311, 1005-1007.
Ericsson, K. A., Krampe, R. T., and Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363-406. Gawande, A. (2007). The checklist. The New Yorker (December 10).
Gettys, C. F., Pliske, R. M., Manning, C., and Casey, J. T. (1987). An evaluation of human act generation performance. Organizational Behavior and Human Decision Processes, 39, 23-51. G
igerenzer, G., and Goldstein, D. G. (1999). Betting on one good reason: The take the best heuristic. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (eds), Simple Heuristics that Make us Smart (pp. 75-95). New York: Oxford University Press.
Gladwell, M. (2005). Blink: The Power of Thinking without Thinking. New York: Little, Brown and Company.
Hammond, J. S., Keeney, R. L., and Raiffa, H. (1997). Smart Choices: A Practical Guide to Making Better Life Decisions. New York: Broadway Books.
Heath, C., Larrick, R. P., and Klayman, J. (1998). Cognitive repairs: How organizations compensate for the shortcomings of individual learners. Research in Organizational Behavior, 20, 1-37.
Herzog, S. M., and Hertwig, R. (2009). The wisdom of many in one mind. Psychological Science, 20, 231-237.
Hirt, E. R., and Markman, K. D. (1995). Multiple explanation: A consider-an-alternative strategy for debiasing judgment. Journal of Personality and Social Psychology, 69, 1069-1086.
Hogarth, R. M. (2001). Educating Intuition. Chicago: University of Chicago.
Hogarth, R. M., and Karelaia, N. (2007). Heuristic and linear models of judgment. Matching rules and environments. Psychological Review, 114, 733-758.
Isaacson, W. (2003). A Benjamin Franklin Reader. New York: Simon & Schuster. K
ahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58, 697-720.
Kahneman, D., and Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, 39, 17-31.
Kray, L. J., and Galinsky, A. D. (2003). The debiasing effect of counterfactual mindsets: Increasing the search for disconfirmatory information in groups. Organizational Bahavior and Human Decision Processes, 91, 61-81.
Larrick, R. P. (2004). Debiasing. In D. J. Koehler and N. Harvey (eds), The Blackwell Handbook of Judgment and Decision Making (pp. 316-337). Malden, MA: Blackwell.
Larrick, R. P., Mannes, A. E., and Soll, J. B. (2009). The wisdom of small crowds. Unpublished manuscript: Duke University.
Lipshitz, R., Klein, G., Orsaanu, J., and Salas, E. (2001). Focus article: Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14, 331-352.
Makridakis, S., Hogarth, R., and Gaba, A. (2009). Dance with Chance: Making Luck Work for You. Oxford: Oneworld Publications.
Miller, C. C. (2008). Decisional comprehensiveness and firm performance: Towards a more complete understanding. Journal of Behavioral Decision Making, 21, 598-620.
Payne, J. W., Bettman, J. R., and Johnson, E. J. (1993). The Adaptive Decision Maker. Cambridge: Cambridge University Press.
Payne, J. W., Samper, A., Bettman, J. R., and Luce, M. F. (2008). Boundary conditions on unconscious thought in complex decision making. Psychological Science, 119, 1118-1123.
Sanbonmatsu, D. M., Posavac, S. S., Kardes, F. R., and Mantel, S. P. (1998). Selective hypothesis testing. Psychonomic Bulletin and Review, 5, 197-220.
Schoemaker, P. J. H. (2004). Forecasting and scenario planning: The challenges of uncertainty and complexity. In D. J. Koehler and N. Harvey (eds), Blackwell Handbook of Judgment and Decision Making (pp. 274-296). Malden, MA: Blackwell.
Soll, J. B., and Klayman, J. (2004). Overconfidence in interval estimates. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 299-314.
Stanovich, K. E. (1999) Who is Rational? Studies of Individual Differences in Reasoning. Mahwah, NJ: Erlbaum.
Surowiecki, J. (2004). The Wisdom of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations. New York: Doubleday.
Tetlock, P. E. (2005). Expert Political Judgment: How Good is It? How Can We Know? Princeton, NJ: Princeton.
Tufte, E. R. (1997). Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire, CT: Graphics Press.
Walton, M. (1990). Deming Management at Work. New York: Putnam.
Wilson, T. D., Lisle, D. J., Schooler, J. W., Hodges, S. D., Kaaren, K. J., and La Fleur, S. J. (1993). Introspecting about reasons can reduce post-choice satisfaction. Personality and Social Psychology Bulletin, 19, 331-339.
Zbaracki, M. J. (1998). The rhetoric and reality of total quality management. Administrative Science Quarterly, 43, 602-636.
Generating alternative causes
Have a group of participants think about the following scenario: “The Delta Corporation is in trouble. One of its best products, the portable z-phone, is not selling well even while related products from Apple and other companies are selling very well.”
Divide the participants into groups of about five. First, have each person work alone to list all the reasons why the product might not be selling. Then have each group combine ideas from all members (discarding those they think are preposterous). Count how many more ideas the group as a whole had as compared to each individual member, including the best member. If there is time, see if the groups combined did better than any one group. Further, there could be a discussion of what actions to take on each of the ideas presented.
Generating objectives
This exercise is modeled after research conducted by Bond et al. (2008). In this exercise, participants are asked to generate objectives for an important decision. If this is conducted in a group, try to identify a decision of common interest and importance. For example, ask participants “What objectives are relevant to you when buying a house?” (Depending on the audience, other topics might include business unit objectives, choosing a major, or choosing a job.)
Have participants independently generate a list of objectives. Then break them into teams to compare lists of objectives. If you wish to conduct a formal exercise, you can have a note-taker list all of the objectives for each team, which will involve grouping and eliminating redundant objectives. Have participants assess the completeness of their own list of objectives - were there important objectives generated by other group members that they overlooked? Expand the conversation by having participants discuss how a list of objectives could be used as a decision tool in an organization.