Happiness Metric

“Massacre at Chios” — Yue Minjun

… you have a Community of Trust and have a common Vision. The team is a mature team with a shared sense of value. The Scrum Team is an Autonomous Team. You are holding regular Sprint Retrospectives to increase velocity and other traditional measures of value and potential to generate value.

✥      ✥      ✥

In reflection and other self-improvement activities, there are generally many ideas for improvement. The heart of Scrum is incremental improvement — what in Scrum's Japanese roots is called kaizen. But you often don’t know in advance which improvement activities will produce great benefits, and which will not.

The pattern One Step at a Time recommends focusing on a single improvement, so that the effects of an improvement activity are clear. But there are many opportunities for improvement, and you need a way to work on the things that are most likely to have the most positive benefit.

It is natural to come up with long lists of things that are supposed to help improve velocity. Because there are so many, it’s possible that you are working on the wrong thing; the improvement selected will not actually improve velocity at all, and if it does, it is likely not the biggest or most important improvement.

People often feel disconnected to these long lists because most people can't manage more than a handful of ideas at a time. Having many options to choose from itself actually changes the emotional state, pushing it towards “negative emotions.” [24] So people will not be motivated to make the items work.

So you need to work on the right improvement, and the team must feel some passion about the improvement.

We commonly believe that people derive great satisfaction from doing their job well. But more importantly, they are often in a good position to understand what things can make them more effective, and what things are standing in their way. Associated with this, most people have a strong sense of responsibility toward their jobs, particularly if they are in an Autonomous Team.

Therefore:

Drive the improvement process with a single, small improvement at a time, chosen through team consensus. Pose a question to the team that helps them reflect on which of the alternatives on the table will best tap into their collective passion, and use the answer to choose the kaizen that will most energize the team. The team commits (to itself) to work on that item in the next Sprint.

It's important to drive to consensus, rather than using majority voting or a pronouncement by the ScrumMaster or Product Owner, to arrive at the decision. Voting and decision by fiat are both forms of control. The operative mechanism here is that the team feels they have control, and anything that dilutes that sense cuts to the core of what makes the pattern work. In the end, this is about tapping into and amplifying the team's sense of autonomy. The decision must tap into the team's passion rather than individuals' willingness to defer to any a priori agreed “decision process” or even to objective value- or business-based indicators alone.

Affective measures such as happiness can and should be an end in themselves, but one also must pay the bills. Most “temperature readings” (affective assessments) of a team’s state attempt to align other values that are indicators of business success. One of the most common is engagement, and there are others as well: [25]

[Work] engagement is a positive, fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption. Rather than a momentary and specific state, engagement refers to a more persistent and pervasive affective-cognitive state that is not focused on any particular object, event, individual, or behavior. Vigor is characterized by high levels of energy and mental resilience while working, the willingness to invest effort in one’s work, and persistence even in the face of difficulties. Dedication refers to being strongly involved in one's work and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge. Absorption, is characterized by being fully concentrated and happily engrossed in one’s work, whereby time passes quickly and one has difficulties with detaching oneself from work.

That is, the team will put its highest level of energy behind an improvement whose implementation engages them the most. This level of dedication in turn benefits both team well-being and business improvement — which are ostensibly the reasons that the team is motivated to work on them.

There are various ways to measure the team's sense of engagement in each kaizen, but teams popularly use a simple subjective five-point scale for “happiness.” You don’t need to be fancy. We believe that a happy team strives to make things even better, and of course a team where nothing is working can't be entirely happy. Happiness is a measurement of overall climate, expectations, and environment, and is a sum of incremental happiness improvements, ([13], p. 88). Happiness may be a measure of engagement but, more importantly, letting the team decide what will improve its happiness sends a message that the team is autonomous, and autonomy has well-established links to a state of emotional well-being. However, even more important than happiness may be autonomy itself, team morale, and, above all else, passion. Autonomy support in workplaces has consistently been related to workplace engagement, productivity, organizational citizenship, and generally a full list of pro-social behaviors. Edward L. Deci and Richard M. Ryan's research into self-determination theory [8] has shown this to be the case across time and cultures.

The pattern is called ‟Happiness Metricˮ for historical reasons, in line with early attempts to identify a reliable emotional predictor of success. While overall happiness correlates to good future performance, modern research shows that ascribing happiness (or disappointment) to any specific change is problematic. And though the evidence is strong that there is a correlation between current happiness and future performance, the arguments for causality are scarce. Measuring (merely) happiness sometimes works — not because anticipation of happiness is causal, but rather because autonomy is being exercised. Henrik Kniberg of Crisp — the company that pioneered the metric in the agile software development space — notes that one of its main benefits is to turn the organization's attention to awareness of itself, and that independent of any numbers, the fact that happiness is measured at all reinforces Crisp's value proposition to all involved. [17] As described in the work of Deci and Ryan, it focuses on positive emotional and personal outcomes, rather than on quantitative measures that are often politically manipulated. Such measures are nonetheless obviously empirical. In practice, teams have been inclined to measure happiness because they feel there is likely a two-way back and forth effect between happiness and improvement efforts. First, this is partly true, but more important, it can never stand alone (see the discussion below). It is perhaps too often accepted because of the truism that ‟no one can argue with happiness;ˮ business and commercial practice have long promised a pot of happiness at the end of every rainbow:

HSBC Bank advertisement, June, 2017

We critically revisit that premise below and implicitly explore what might lie behind the masks. The question therefore can better be formulated to tap into a broad sense of value and an emotional spectrum beyond happiness, such as in:

What can we do next Sprint that will both improve our team as an even greater place for us all to work, and improve our product so that the market realizes even greater value from it?

The team should reflect on maintaining a balance between these two concerns. A simpler variant that builds on passion (see below) is:

What are we most excited about, as a team, to do next Sprint to improve our process?

While tapping into the team's shared energy for an idea, temper the consideration with sober feasibility. The team should make testing criteria clear and should estimate the work necessary to achieve this kaizen. As in Scrumming the Scrum, account for the estimate by packaging the kaizen as an estimated PBI on top of the Product Backlog.

✥      ✥      ✥

The team's kaizen efforts will move in a direction that reflects the passion of the team, instead of deferring to business measures or objective criteria alone. In healthy teams, this passion is linked to the collective notion of Greatest Value that precipitates from the Vision. To tap into that passion means shifting focus from the rationally dominated to the affectively dominated: the realm of feelings. There is a chorus of voices speaking to the fidelity of emotional measures, among them Jerry Weinberg and his timeless saying, “Feelings are facts,” [21] and there is a good body of medical research substantiating this (see [22]). Emotional cues telegraph the deeper concerns that matter. Dan Pink proposes that the triad of autonomy, mastery, and purpose is fundamental to the engagement fueling business success. [7] Coincidentally (with no reference to Pink), Jeff Sutherland notes that autonomy, mastery, and purpose are not only the foundations of great teams, but are also at the root of happiness ([9], p. 153). He offers anticipated happiness as his chosen emotional indicator.

Another key to this pattern's success is that framing the chosen kaizen in terms of what the team thinks will be beneficial affirms to the team that it really is an Autonomous Team. It is likely that if one measures the success of a team by how it will make them feel better, it will gain a sense control over its fate and a heightened sense of autonomy. This tacit benefit alone may contribute as much to the emotional state of the team as do the prospects for the improvement results themselves.

The desired improvement might become part of the Sprint Goal, in which case this pattern can help create the Sprint Goal. A series of Sprint Goals line up with Pink's broader notion of purpose. Etzioni [19] puts happiness in conflict with affirmation or doing that which is in line with the team’s moral commitments, and a proper sense of purpose reflects that the team is doing something morally profound.

Is happiness the right metric? The research shows that happiness does indeed measure something, and it is widely held that happiness links to engagement, which in turn is widely believed to correlate to good team performance. It turns out that focusing on the happiness of the team indeed helps the team uncover issues standing in the way not only of happiness as an end in its own right, but of effectiveness as well. The argument for the Happiness Metric is that people feel more personally connected to and committed to the improvement (engagement) and team performance correspondingly improves. It is widely believed that this is because people derive great satisfaction from doing their job well. However, the research shows that in fact the opposite is more often true: established happiness leads to good performance more than good performance makes employees happy. In [6], the authors discuss the oft-cited Gallup study that suggests a correlation between happiness and good performance, but note that the causality is reversed from how we usually see it:

The effort to use happiness as a measure of a society’s productivity on a macro scale is paralleled and bolstered by research on the micro level, which shows that being happy at work actually makes individuals more productive. In a recent study done for Gallup using a longitudinal database of 2,178 business units in 10 large organizations, the researcher James Harter “found evidence supporting the causal impact of employee perceptions on bottom-line measures” like customer loyalty, employee retention, revenue, sales and profit. In a related finding, the Gallup-Healthways well-being index showed earlier this year that Americans of all ages and income levels felt less happy at work and more disengaged from what they do than ever before. Gallup found that this disengagement correlated with lower productivity and poorer health outcomes and cost companies an estimated $300 billion annually. (Emphasis ours) [6]

So it is widely held that happiness is a leading indicator of good performance. This is sometimes true, yet correlation is not causality. The passion and engagement that bode for good performance can evidence themselves in a number of emotions: excitement, confidence, self-assuredness, anticipation, determination, optimism, and sometimes happiness. But they may also trigger other emotions for which we do not hold a positive association, such as desperation or righteous anger. Weinberg notes that there is no such thing as a “bad emotion,” and the broad research shows that “negative emotions” are key to well-being (see, e.g., [15]). In one HBR article, we find that grumpiness, rather than happiness, bodes for good performance and that, again, the causality is not what one might think it is: [10]

. . . in a study we conducted on symphonies, we actually found that grumpy orchestras played together slightly better than orchestras in which all the musicians were really quite happy.

That's because the cause-and-effect is the reverse of what people believe: When we're productive and we've done everything good together (and are recognized for it), we feel satisfied, not the other way around. In other words, the mood of the orchestra members after a performance says more about how well they did than the mood beforehand.

So not only is happiness insufficient for success, it is also unnecessary. Happiness is not always a leading indicator of success, which strongly suggests that anticipated happiness is even less so. The Danes — one of the most efficient economies in the world — were gauged as the happiest people on Earth for years, and much of the theory prognosticates that it because their expectations for the future were so low. [12] (In fact, the Danes would not say that they were happy, but would use the untranslatable word tilfreds — which might best be rendered as “content.”)

In the publication of a study done at Deloitte, [2] the authors attempt to typify the emotional roots of great performance through a quote from Roosevelt, [3] who suggests that the path to achievement is hardly one of happiness:

The credit belongs to the man who is actually in the arena, whose face is marred . . . ; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.

Sutherland, by coincidence, has also used this quote, in the emotive context of a team that was “frantic” and “nervous” with “tremendous time pressure” and “skeptical” stakeholders who expected yet another in a series of failures — and “happy” is conspicuous by its absence. ([9], pp. 17-18) The quote was apparently chosen to communicate this demeanor of the team. But one can sense drive, passion, and a sense of team identity in both the quote and in the example Sutherland relates.

Focusing on happiness may paradoxically lead to a self-fulfilling prophecy and a “happy bubble” (see Beyond the Happy Bubble). Peter N. Stearns notes: ([14], p. 109)

[A]lthough the most obvious drawback of the emphasis on happiness involves the gaps with reality that can, paradoxically, create their own discontents, there’s also the risk that people will fail to explore reasons for dissatisfaction because of pressure to exhibit good cheer. . . Those risks suggest the need to cut through the pervasive happiness rhetoric at certain points.

Happiness, therefore, may be the wrong metric, also because it fails to predict well and works at the wrong level of granularity. One can be temporarily encouraged or excited about the prospects of the success of some improvement, but happiness comes from less situational drivers. A host of research (summarized in [12], p. 240) suggests that “objective circumstances, . . . and life events” contribute to no more than about 8% and certainly no more than 15% of the variance in happiness. Further, “people are not very good at predicting what will make them happy and how long that happiness will last.” ([13], p. 86) This implies that it doesn't work to tie the benefit of potential improvement to an anticipated sense of happiness. In no case does the literature offer evidence that any gradient of happiness could be used to select the best from among alternative improvements or mitigations.

Last, happiness is more the result of an integration over many successive, small improvements than of any single change, and an improvement in happiness rarely can be tied to any specific, single change. ([13], p. 88) Henrik Kniberg notes: [17]

We've seen many examples of the metric influencing our actions, and our actions influencing the metric (but not on a day-by-day basis, more like a few times per year). . . Interestingly enough our average happiness is very stable and boring, it hovers around 4.0 (+/- 0.2) most of the time. And in that sense the average happiness is pretty useless data, there are no interesting trends to follow, except some things like that the happiness tends to gradually degrade and then make a jump back up after our unconferences. The ROI of getting everyone together was directly measurable :)

Some even argue that happiness is genetically linked, and that “urging a person to become happier is like insisting she become taller.” ([14], p. 108) These insights suggest that the use of happiness at all is an unreliable indicator of any property, emotional or otherwise, of any single change.

Putting the team's fate in its own hand gives the team a stronger sense of autonomy, but autonomy alone won't carry the day. Therefore, the measure needs a connection to some purpose that transcends the individual. The team members must share some sense of value and be focused on the whole, and an outwardly-directed sense of Greatest Value. Consider this case study contributed by John Hayes for one of his teams. Happiness was measured in terms of a spectrum that ranged from loving their job and not wanting to look elsewhere, to desperately seeking alternative employment. They started doing Scrum and started moving towards a Collocated Team. The team was measurably unhappy with this, as each team member valued being able to work from home. Happiness drove to an all-time low. So in Sprint 14 people were allowed to work individually, at home, in silos. (The gap in the data comes from two Sprints that weren't measured, because the team abandoned their faith in the Happiness Metric while their impatience with collocation grew.)

Team Happiness (Likelihood to stay working at the same job)

Velocity went up for a short period, but then curiously returned about to the level it had achieved before the change. Note, however, that happiness sustained a high level and even increased slightly over time.

Team Velocity (time-smoothed average)

Happiness was measured as an end in its own right, with no consideration for engagement or transcendent purpose, which made it a bad choice from a business perspective. So while happiness seems to have a high correlation with performance, it is not sufficient to support any non-emotive business goal. The question to the team must be formulated in terms of what we often call the two products of a Scrum effort: the team itself, and the product that the team delivers to the market. The broad research bears out the danger of measuring happiness alone: [1]

Many companies are using misleading data to define how good their culture is. If they only ask their employees to rate their happiness on a scale from 1-10, they are leaving a lot of missed opportunities on the table.

We have explored Pink’s admonishment for autonomy and mastery — what about purpose? Etzioni [19] warns us that happiness is hedonistic, and that to be attentive to the kind of transcendent value (as in Greatest Value) and vision that goes beyond the individual, we must instead attend to what is morally profound. He notes that: “Subjective well-being is surely a much more meaningful and richer concept than that of happiness.” That begs a different metric:

This views the person as being subject to an irreconcilable conflict between the quest for happiness (of one kind or another) and the quest to live up to their moral values, with the completion of the latter resulting in a sense of affirmation.

This affirmation is much more in line with the pursuit of Greatest Value that happiness is, exemplified in the mission statement of Toyota USA when they set up the truck assembly plant in Nummi, California: [20]

  • As an American company, contribute to the economic growth of the community and the United States.
  • As an independent company, contribute to the stability and well-being of team members
  • As a Toyota group company, contribute to the overall growth of Toyota by adding value to our customers

It has always been a puzzle when doing Scrum interventions in companies whose products are bad, or who have been forcibly placed under administrative oversight because of poor performance, about what motivates people to stay there. We started asking, and found that it's because people “enjoy the challenge,” or because the pay is good relative to other companies in the same business sector. Surprisingly few felt trapped in their jobs. The people are happy enough not to switch jobs. Yet their products are buggy and late and the business continuously interferes in development's affairs. The workers are often out of touch with their end users and feel no day-to-day consequences from lack of improvement. It begs the questions: What's important to measure? What does happiness really indicate? The answer in these environments is often, “I like the challenge” — which indeed contributes to satisfaction and maybe even happiness, which benefits only the individual and which falls far short of affirmation.

Christiaan Verweijs observes: [4]

I strongly believe that in a cohesive, well-running team, people are willing to go the extra mile even if it makes them (a bit) unhappy for the duration of the task. They will accept this for the greater good.

Happiness can be an indicator of moving towards Greatest Value, if the team is circumspect and honest with itself. But, by the reasoning of the Gallup study, happiness is more an enabler for future results than an indicator of process or product qualities of intrinsic value in the current work setting. Happiness is good for its own sake. Google knows this and has a whole organization supporting a mindfulness program for its employees. The book describing this program starts with a description of “the happiest man in the world.” [18] Its author, Chade-Meng Tan, is Google's “jolly good fellow.” The book is credited with no small part of Google's success and its techniques are supported with impressive scientifically validated results. Create the happiness first, then select from potential alternative courses of action. The fact that the team is set up to chart a direction that optimizes their happiness may, in itself, be enough to engender a sense of happiness. It may remove the threat of willful business decisions trading off business benefits against quality of work life. These are the threats behind fear in the workplace. Deming says that the first thing you do is to drive out fear. [11] Letting a team decide what it is going to do to increase its assuredness and confidence (perhaps instead of happiness — i.e., autonomous decision-making) may be one key to this first step advocated by Deming.

While happiness and engagement (which may be closely linked) are important, the notion of passion in the work place drives even deeper. The Deloitte study concluded that increased engagement (as measured by happiness) leads to only single spurts of improvement rather than sustained improvement. Furthermore, they hold that engagement is not sufficient (and may not be necessary) to improve performance levels:

The concept of worker passion, which we describe as the “passion of the Explorer,” is different from engagement. Employee engagement is typically defined by how happy workers are with their work setting, coworkers, organization-wide programs, and their overall treatment by their employer. Employee engagement is important, and improving it typically will give a firm a bump in performance. But engagement is often a one-time bump; employees move from unhappy to happy, bring a better attitude to work, and possibly take fewer sick days. However, workers who are merely engaged won’t actively seek to achieve higher performance levels, to the benefit of self and firm; passionate workers will, though. (Emphasis ours) [2]

Note that the example from John Hayes above evidenced exactly what the Deloitte study found: that an improvement linking happiness to the team's engagement in the improvement led to a single bump in performance. This property of passion recurs in several other studies. Spreitzer and Porath call it vitality, [16] which they complement with continuous learning. They describe how the two together lead to a sustainable lustre in performance.

Christiaan Verwijs offers team morale as an alternative to happiness and pointedly describes why it is better than happiness: it is team-oriented, less susceptible to mood, and not as biased as happiness. His research is based on data from over 10,000 subjects and over 2,300 teams. He draws his arguments from research in military psychology texts on team cohesion and effectiveness, which define team morale as follows:

(Team) Morale is the enthusiasm and persistence with which a member of a team engages in the prescribed activities of that group  [5]

This definition closely matches the findings of the Deloitte study. If teams are autonomous (as most modern militaries have been striving for during the past 40 years) then this approach supports the effectiveness of measuring happiness, to the degree that such measurement suggests that the team is able to navigate its own way to improvement.

Verwijs has research data that strongly correlates morale with autonomy and purpose.

The Happiness Metric can help prevent burnout. Burnout occurs when people work long hours or spend much mental energy over an extended period without any respite. They just get tired of the pace. Burnout can kill productivity through less work later (“. . . there will be more or less an hour of undertime for every hour of overtime” — Tom DeMarco and Tim Lister [23]) or by people leaving to find a saner environment. If burnout is threatening, someone will likely request to “stop the insane work hours” as the upcoming kaizen.

Some people (such as stereotypical old-time managers) may fear that the team could “game” the system; deciding that their happiness could best be improved by taking every Friday off, for example. Of course this is possible. Like all aspects of team autonomy, one must trust the team to follow The Spirit of the Game. Trust the team, but verify: the ScrumMaster should ensure that the goals are within the value proposition. (And if you don’t trust the team, or if the team violates The Spirit of the Game, you have much bigger problems on your hands!)


[1] Jacob Shriar, “Why Measuring Employee Happiness Is A Huge Mistake.” 2 August 2015, https://www.officevibe.com/blog/measuring-employee-happiness-huge-mistake#fn-5054781-1, accessed 17 February 2017.

[2] John Hagel, John Seely Brown, Alok Ranjan, and Daniel Byler. “Passion at work: Cultivating worker passion as a cornerstone of talent development.” Deloitte University Press, 7 October 2014, https://dupress.deloitte.com/dup-us-en/topics/talent/worker-passion-employee-behavior.html.

[3] Theodore Roosevelt, excerpt from “Citizenship in a republic” (speech delivered at the Sorbonne, Paris, France, April 23, 1910).

[4] Christiaan Verwijs, “Agile Teams: Don't use happiness metrics, measure Team Morale.” http://blog.agilistic.nl/agile-teams-dont-use-happiness-metrics-measure-team-morale/.

[5] Manning, F. J. Morale, Cohesion, and Esprit de Corps. In Gal, R. & Mangelsdorff, A. D. (Eds.), Handbook of Military Psychology (pp. 453-470). Chichester: John Wiley & Sons Ltd., 1991.

[6] Korn Ferry Institute, “The Happiness Metric.” Briefings Magazine, 11 May 2012, http://www.kornferry.com/institute/423-the-happiness-metric, accessed 17 February 2017.

[7] Daniel Pink, Drive: The amazing truth about what motivates us, Riverhead Books, 2011.

[8] Edward L. Deci and Richard M. Ryan. Self-determination Theory: A Macrotheory Of Human Motivation, Development, And Health. Canadian Psychology/Psychologie Canadienne 49(3), 2008, pp. 182-185.

[9] Jeff Sutherland, Scrum: The art of doing twice the work in half the time. New York: Crown Business, 2014.

[10] Diane Coutu, Why Teams Don't Work. Harvard Business Review, May 2009, https://hbr.org/2009/05/why-teams-dont-work.

[11] W. Edwards Deming. Out of the Crisis. MIT Press, 2017, Chapter 3, “Disease and Obstacles.”

[12] Sonja Lyubomirsky. Why are some people happier than others? The role of cognitive and motivational process in well-being. Am Psychol 2001;vol. 56, pp. 239-49.

[13] Daniel Gilbert. ‟The Science Behind the Smile.ˮ Harvard Business Review, January / February 2012, pp. 85-90.

[14] Peter N. Stearns. ‟The History of Happiness.ˮ Harvard Business Review, January / February 2012, pp. 104-109.

[15] Tori Rodriguez. ‟Taking the Bad with the Good.ˮ Scientific American Mind 24(2), May 2013, pp. 26-27.

[16] Gretchen Spreitzer and Christine Porath. ‟Creating Sustainable Performance.ˮ Harvard Business Review, January / February 2012, pp. 93-99.

[17] Henrik Kniberg. Personal conversation of 29 June 2017.

[18] Chade-Meng Tan et al. Search Inside Yourself: The Unexpected Path to Achieving Success, Happiness (and World Peace). HarperOne, 2012.

[19] Amitai Etzioni. Happiness Is the Wrong Metric. Society 53(3), March 2016, pp. 246-257.

[20] Jeffrey Liker. The Toyota Way: Fourteen Management Principles from the World's Greatest Manufacturer. New York: McGraw-Hill, 2003, p. 80.

[21] Donald Gause and Jerry Weinberg. Exploring Requirements 2: First Steps Into Design. Weinberg & Weinberg, 2011, Chapter 21, 21.2.4.

[22] M. A. Hoogerwerf et al. “Feelings are facts”: Illness perceptions in patients with lung cancer. Respiratory Medicine 106(8), August 2012, pp. 1170-1176.

[23] Tom DeMarco and Tim Lister. Peopleware, Addison-Wesley, 2015, p. 15.

[24] Barry Schwartz. The Tyranny of Choice. Scientific American 290(4), April 2004, pp: 71-75.

[25] Schaufeli, W. B., Salanova, M., Gonzalez-Roma. V. & Bakker, A. B. The measurement of engagement and burnout and: A confirmative analytic approach. Journal of Happiness Studies, 3, 2002, 71-92.


Picture: “Massacre at Chios”, by Yue Minjun, 1994.