American paleontologist Stephen Jay Gould was one of the most popular and highly regarded science writers of the late 20th century. All 300 of the monthly columns that he wrote for Natural History magazine were collected in books, and his numerous awards include the National Book Award (1981), the Distinguished Scientist Award (1997), Humanist of the Year (2001), and the Darwin-Wallace Medal (2008).
Gould was also a devoted baseball fan. In his essay "The Streak of Streaks" (first published in The New York Review of Books in 1988), he explored the significance of Joe Dimaggio's 56-game hitting streak in 1941, generally considered to be "the greatest accomplishment in the history of baseball, if not all modern sport." In this essay, Gould relies on examples and statistics to explain--and debunk--the popular belief known as "hot hands," a version of the gambler's fallacy.
My father was a court stenographer. At his less than princely salary, we watched Yankee games from the bleachers or high in the third deck. But one of the judges had season tickets, so we occasionally sat in the lower boxes when hizzoner couldn’t attend. One afternoon, while DiMaggio was going 0 for 4 against, of all people, the lowly St. Louis Browns (now the even lowlier Baltimore Orioles), the great man fouled one in our direction. “Catch it, Dad,” I screamed. “You never get them,” he replied, but stuck up his hand like the Statue of Liberty—and the ball fell right in. I mailed it to DiMaggio, and, bless him, he actually sent the ball back, signed and in a box marked “insured.” Insured, that is, to make me the envy of the neighborhood, and DiMaggio the model and hero of my life.
I met DiMaggio a few years ago on a small playing field at the Presidio of San Francisco. My son, wearing DiMaggio’s old number 5 on his Little League jersey, accompanied me, exactly one generation after my father caught that ball. DiMaggio gave him a pointer or two on batting and then signed a ball for him. One generation passeth away, and another generation cometh: but the earth abideth forever.
My son, uncoached by Dad, and given the chance that comes but once in a lifetime, asked DiMaggio as his only query about life and career: “Suppose you had walked every time up during one game of your fifty-six–game hitting streak? Would the streak have been over?” DiMaggio replied that, under 1941 rules, the streak would have ended, but that this unfair statute has since been revised, and such a game would not count today.
My son’s choice for a single question tells us something vital about the nature of legend. A man may labor for a professional lifetime, especially in sport or in battle, but posterity needs a single transcendant event to fix him in permanent memory. Every hero must be a Wellington on the right side of his personal Waterloo; generality of excellence is too diffuse. The unambiguous factuality of a single achievement is adamantine. Detractors can argue forever about the general tenor of your life and works, but they can never erase a great event.
In 1941, as I gestated in my mother’s womb, Joe DiMaggio got at least one hit in each of fifty-six successive games. Most records are only incrementally superior to runners-up; Roger Maris hit sixty-one homers in 1961, but Babe Ruth hit sixty in 1927 and fifty-nine in 1921, while Hank Greenberg (1938) and Jimmy Foxx (1932) both hit fifty-eight. But DiMaggio’s fifty-six–game hitting streak is ridiculously and almost unreachably far from all challengers (Wee Willie Keeler and Peter Rose, both with forty-four, come second). Among sabermetricians1—a contentious lot not known for agreement about anything—we find virtual consensus that DiMaggio’s fifty-six–game hitting streak is the greatest accomplishment in the history of baseball, if not all modern sport.
The reasons for this respect are not far to seek. Single moments of unexpected supremacy—Johnny Vander Meer’s back-to-back no-hitters of 1938, Don Larsen’s perfect game in the 1956 World Series—can occur at any time to almost anybody, and have an irreducibly capricious character. Achievements of a full season—Maris’s sixty-one homers, Ted Williams’s batting average of .406, also posted in 1941 and not equaled since—have a certain overall majesty, but they don’t demand unfailing consistency every single day; you can slump for a while, so long as your average holds. But a streak must be absolutely exceptionless; you are not allowed a single day of subpar play, or even bad luck. You bat only four or five times in an average game. Sometimes two or three of these efforts yield walks, and you get only one or two shots at a hit. Moreover, as tension mounts and notice increases, your life becomes unbearable. Reporters dog your every step; fans are even more intrusive than usual (one stole DiMaggio’s favorite bat right in the middle of his streak). You cannot make a single mistake.
Thus Joe DiMaggio’s fifty-six–game hitting streak is both the greatest factual achievement in the history of baseball and a principal icon of American mythology. What shall we do with such a central item of our cultural history? Michael Seidel’s happy response is a book devoted not to generalities or implications of the streak—many have done this, too many times—but to day-by-day details of how a man gets from one to fifty-six with no misses in between. This book chronicles the intricate factual events of DiMaggio’s achievement, and pays the best kind of proper respect, while providing the right sort of description. I shall return to Seidel, but first let me illustrate another approach to such an icon.
Statistics and mythology may seem the most unlikely bedfellows. How can we quantify Caruso or measure Middlemarch? But if God could mete out heaven with the span (Isaiah 40:12), perhaps we can say something useful about hitting streaks. The statistics of “runs,” defined as continuous series of good or bad results (including baseball’s streaks and slumps), is a well-developed branch of the profession, and can yield clear—but wildly counterintuitive—results. (The fact that we find these conclusions so surprising is the key to appreciating DiMaggio’s achievement, the point of this article, and the gateway to an important insight about the human mind.)
Start with a phenomenon that nearly everyone both accepts and considers well understood—”hot hands” in basketball. Now and then, someone just gets hot, and can’t be stopped. Basket after basket falls in—or out as with “cold hands,” when a man can’t buy a bucket for love or money (choose your cliché). The reason for this phenomenon is clear enough; it lies embodied in the maxim: “When you’re hot, you’re hot; and when you’re not, you’re not.” You get that touch, build confidence; all nervousness fades, you find your rhythm; swish, swish, swish. Or you miss a few, get rattled, endure the booing, experience despair; hands start shaking and you realize that you shoulda stood in bed.
Everybody knows about hot hands. The only problem is that no such phenomenon exists. The Stanford psychologist Amos Tversky studied every basket made by the Philadelphia 76ers for more than a season. He found, first of all, that probabilities of making a second basket did not rise following a successful shot. Moreover, the number of “runs,” or baskets in succession, was no greater than what a standard random, or coin-tossing, model would predict. (If the chance of making each basket is 0.5, for example, a reasonable value for good shooters, five hits in a row will occur, on average, once in thirty-two sequences—just as you can expect to toss five successive heads about once in thirty-two times, or 0.55.)
Of course Larry Bird, the great forward of the Boston Celtics, will have more sequences of five than Joe Airball—but not because he has greater will or gets in that magic rhythm more often. Larry has longer runs because his average success rate is so much higher, and random models predict more frequent and longer sequences. If Larry shoots field goals at 0.6 probability of success, he will get five in a row about once every thirteen sequences (0.65). If Joe, by contrast, shoots only 0.3, he will get his five straight only about once in 412 times. In other words, we need no special explanation for the apparent pattern of long runs. There is no ineffable “causality of circumstance” (if I may call it that), no definite reason born of the particulars that make for heroic myths—courage in the clinch, strength in adversity, etc. You only have to know a person’s ordinary play in order to predict his sequences. (I rather suspect that we are convinced of the contrary not only because we need myths so badly, but also because we remember the successes and simply allow the failures to fade from memory. More on this later.) But how does this revisionist pessimism work for baseball?
My colleague Ed Purcell, Nobel laureate in physics but, for purposes of this subject, just another baseball fan,2 has done a comprehensive study of all baseball streak and slump records. His firm conclusion is easily and swiftly summarized. Nothing ever happened in baseball above and beyond the frequency predicted by coin-tossing models. The longest runs of wins or losses are as long as they should be, and occur about as often as they ought to. Even the hapless Orioles, at 0 and 21 to start this season, only fell victim to the laws of probability (and not to the vengeful God of racism, out to punish major league baseball’s only black manager).
But “treasure your exceptions,” as the old motto goes. There is one major exception, and absolutely only one—one sequence so many standard deviations above the expected distribution that it should not have occurred at all. Joe DiMaggio’s fifty-six–game hitting streak in 1941. The intuition of baseball aficionados has been vindicated. Purcell calculated that to make it likely (probability greater than 50 percent) that a run of even fifty games will occur once in the history of baseball up to now (and fifty-six is a lot more than fifty in this kind of league), baseball’s rosters would have to include either four lifetime .400 batters or fifty-two lifetime .350 batters over careers of one thousand games. In actuality, only three men have lifetime batting averages in excess of .350, and no one is anywhere near .400 (Ty Cobb at .367, Rogers Hornsby at .358, and Shoeless Joe Jackson at .356). DiMaggio’s streak is the most extraordinary thing that ever happened in American sports. He sits on the shoulders of two bearers—mythology and science. For Joe DiMaggio accomplished what no other ballplayer has done. He beat the hardest taskmaster of all, a woman who makes Nolan Ryan’s fastball look like a cantaloupe in slow motion—Lady Luck.
Seidel’s book succeeds with a simple and honorable premise. The streak itself is such a good story, such an important event in our cultural history, that the day-by-day chronicle will shape a bare sequence into a wonderful drama with beginning, middle, and end. And so we move from the early days of DiMaggio’s streak, when no one realized that anything of interest was happening; through the excruciating middle games when George Sisler’s modern record of forty-one, then Keeler’s all-time mark of forty-four, were approached and broken; to later times of pleasure and coasting when DiMaggio was only smashing his own record, set the day before; and to the final, fateful game fifty-seven, when Ken Keltner made two great plays at third base and lost DiMaggio the prospect of a lifetime advertising contract with the Heinz ketchup company.
But just as baseball, at least in our metaphors, is so much more than a game, Seidel has written more than a sports book. Seidel is professor of literature at Columbia University, and Streak belongs to a growing genre of baseball books written by, if you will pardon my citation of the last footnote, “pointy-heads” (no offense intended since I, alas, am one). Baseball has long enjoyed a distinguished literature, from Ring Lardner to the incomparable Roger Angell—and I have seen no satisfactory resolution for the old puzzle of why baseball, but no other sport, has attracted some of America’s finest writers. But the new genre is quite different—serious, scholarly books treating baseball as something that might even get you tenure at a major university (as something other than athletic coach): Jules Tygiel’s Baseball’s Great Experiment,3 on Jackie Robinson and the racial integration of the game; or Charles C. Alexander’s Ty Cobb,4 a sociology of a time as well as a biography of the greatest and nastiest player of them all.
Seidel has tried to place DiMaggio’s feat in a larger setting of history and culture by weaving in and around his fifty-six-game saga the chronicle of other events during the summer of 1941. There is no tendentiousness here, no attempt to find deep meaning, no theory about baseball imitating or reflecting anything about “real” life. Instead, Seidel presents a sensitive and judicious selection of surrounding events simply, I think, to provide the background for a genuine legend. (As my only mild criticism of the book: I felt that this strategy sometimes smacks a bit of reading old newspapers and listing the main events in order.) While DiMaggio hit away, Rudolf Hess parachuted into Scotland, Hitler invaded Russia, and Roosevelt maneuvered America toward its inevitable involvement, as Charles Lindbergh toured the country in the losing cause of an isolationism that, for his part at least, conveyed more than a faint odor of pro-German sympathy. On the evening after DiMaggio hit safely in game twelve, the nighttime contest between the New York Giants and the Boston Braves was halted for forty-five minutes to broadcast FDR’s “unlimited emergency” speech over the loudspeakers. (After paying ransom to the aforementioned thief, DiMaggio auctioned off his streak bat to benefit the USO.) In the meantime, Citizen Kane played its initial run, commercial television made its first broadcasts in New York City, and the Grand Central Red Cap Barbershop Quartet won the citywide contest sponsored by the New York Parks Department, only to face the indignity of disqualification at the nationals in Saint Louis. The Red Caps were black.
With apologies to Seidel (for he was probably sandbagged by his publishers on this), I must mention one funny error. The caption to a photo reads: “Ted Williams as he looked in 1941 when he hit .406.” But the picture, unless I need a very peculiar pair of glasses, shows Phil Rizzuto. Now you couldn’t find two more different people. Williams was tall, thin, taciturn, cold, the finest hitting machine since Ty Cobb, but not exactly perfect in the outfield. Rizzuto was just the opposite: short, stocky, convivial, not much with the stick, but the finest short-stop in the league. Moreover, Rizzuto played with DiMaggio on the Yankees, Williams for their archrivals, the Boston Red Sox. But then, an even more amusing mixup once appeared in the “errata” section of The New York Times: “The photo that appeared yesterday on page forty-one, labeled as the sun, was the moon.”
Seidel’s book will help us to treasure DiMaggio’s achievement by bringing together the details of a genuine legend. But a larger issue lies behind basic documentation and simple appreciation. For we don’t understand the truly special character of DiMaggio’s record because we are so poorly equipped, whether by habits of culture or by our modes of cognition, to grasp the workings of random processes and patterning in nature.
That old Persian tentmaker, Omar Khayyám, understood the quandary of our lives:
Into this Universe, and Why not knowing,
Nor Whence, like Water willy-nilly flowing; And out of it, as Wind along the Waste,
I know not Whither, willy-nilly blowing.
But we cannot bear it. We must have comforting answers. We see pattern, for pattern surely exists, even in a purely random world. (Only a highly nonrandom universe could possibly cancel out the clumping that we perceive as pattern. We think we see constellations because the stars are dispersed at random in the heavens, and therefore clump in our sight.) Our error lies not in the perception of pattern but in automatically imbuing pattern with meaning, especially with meaning that can bring us comfort, or dispel confusion. Again, Omar took the more honest approach:
Ah, love! could you and I with Fate conspire
To grasp this sorry Scheme of Things entire, Would not we shatter it to bits—and then
Re-mould it nearer to the Heart’s Desire!
We, instead, have tried to impose that “heart’s desire” upon the actual earth and its largely random patterns:
All Nature is but Art, unknown to thee;
All Chance, Direction, which thou canst not see;
All Discord, Harmony not under- stood;
All partial Evil, universal Good.
(Alexander Pope, Essay on Man, end of Epistle 1)
Sorry to wax so poetic and tendentious about something that leads back to DiMaggio’s hitting streak (pointyheadedness in action, I suppose), but this broader setting is the source of our misinterpretation. We believe in “hot hands” because we must impart meaning to a pattern—and we like meanings that tell stories about heroism, valor, and excellence. We believe that long streaks and slumps must have direct causes internal to the sequence itself, and we have no feel for the frequency and length of sequences in random data. Thus, while we understand that DiMaggio’s hitting streak was the longest ever, we don’t appreciate its truly special character because we view all the others as equally patterned by cause, only a little shorter. We distinguish DiMaggio’s feat merely by quantity along a continuum of courage; we should, instead, view his fifty-six–game hitting streak as a unique assault upon the otherwise unblemished record of Dame Probability.
Amos Tversky, who studied “hot hands,” has performed a series of elegant psychological experiments with Daniel Kahneman.5 These long-term studies have provided our finest insight into “natural reasoning” and its curious departure from logical truth. To cite an example, they construct a fictional description of a young woman: “Linda is thirty-one years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.” Subjects are then given a list of hypothetical statements about Linda: they must rank these in order of presumed likelihood, most to least probable. Tversky and Kahneman list eight statements, but five are a blind, and only three make up the true experiment:
Linda is active in the feminist movement;
Linda is a bank teller;
Linda is a bank teller and is active in the feminist movement.
Now it simply must be true that the third statement is least likely, since any conjunction has to be less probable than either of its parts considered separately. Everybody can understand this when the principle is explained explicitly and patiently. But all groups of subjects, sophisticated students who ought to understand logic and probability as well as folks off the street corner, rank the last statement as more probable than the second. (I am particularly fond of this example because I know that the third statement is least probable, yet a little homunculus in my head continues to jump up and down, shouting at me—”but she can’t just be a bank teller; read the description.”)
Why do we so consistently make this simple logical error? Tversky and Kahneman argue, correctly I think, that our minds are not built (for whatever reason) to work by the rules of probability, though these rules clearly govern our universe. We do something else that usually serves us well, but fails in crucial instances: we “match to type.” We abstract what we consider the “essence” of an entity, and then arrange our judgments by their degree of similarity to this assumed type. Since we are given a “type” for Linda that implies feminism, but definitely not a bank job, we rank any statement matching the type as more probable than another that only contains material contrary to the type. This propensity may help us to understand an entire range of human preferences, from Plato’s theory of form to modern stereotyping of race or gender.
We might also understand the world better, and free ourselves of unseemly prejudice, if we properly grasped the workings of probability and its inexorable hold, through laws of logic, upon much of nature’s pattern. “Matching to type” is one common error; failure to understand random patterning in streaks and slumps is another—hence Tversky’s study of both the fictional Linda and the 76ers’ baskets. Our failure to appreciate the uniqueness of DiMaggio’s streak derives from the same unnatural and uncomfortable relationship that we maintain with probability. (If we understood Lady Luck better, Las Vegas might still be a roadstop in the desert, and Nancy Reagan might not have a friend in San Francisco.)
My favorite illustration of this basic misunderstanding, as applied to DiMaggio’s hitting streak, appeared in a recent article by baseball writer John Holway, “A Little Help from his Friends,” and subtitled “Hits or Hype in ‘41” (Sports Heritage, November/December, 1987). Holway points out that five of DiMaggio’s successes were narrow escapes and lucky breaks. He received two benefits-of-the-doubt from official scorers on plays that might have been judged as errors. In each of two games, his only hit was a cheapie. (In game sixteen, a ball dropped untouched in the outfield and had to be ruled a hit, even though the ball could have been caught, had it not been misjudged; in game fifty-four, DiMaggio dribbled one down the third base line, easily beating the throw because the third baseman, expecting the usual, was playing far back.) The fifth incident is an ofttold tale, perhaps the most interesting story of the streak. In game thirty-eight, DiMaggio was 0 for 3 going into the last inning. Scheduled to bat fourth, he might have been denied a chance to hit at all. Johnny Sturm popped up to begin the inning, but Red Rolfe then walked. Slugger Tommy Henrich, up next, was suddenly swept with a premonitory fear: suppose I ground into a double play and end the inning. An elegant solution immediately occurred to him: why not bunt (an odd strategy for a power hitter)? Henrich laid down a beauty; DiMaggio, up next, promptly drilled a double to left.
Holway’s account is interesting, but his premise is entirely, almost preciously, wrong. First of all, none of the five incidents represents an egregious miscall. The two hits were less than elegant, but they were undoubtedly legitimate; the two boosts from official scorers were close calls on judgment plays, not gifts. As for Henrich, I can only repeat manager Joe McCarthy’s comment when Tommy asked him for permission to bunt: “Yeah, that’s a good idea.” Not a terrible strategy either—to put a man into scoring position for an insurance run when you’re up 3–1.
But these details do not touch the main point—Holway’s premise is false because he accepts the conventional mythology about long sequences. He believes that streaks are unbroken runs of causal courage—so that any prolongation by hook-or-crook is an outrage against the deep meaning of the phenomenon. But extended sequences are no such thing. Long streaks always are, and must be, a matter of extraordinary luck imposed upon great skill. Please don’t make the vulgar mistake of thinking that Purcell or Tversky or I or anyone else would attribute a long streak to “just luck”—as though everyone’s chances are exactly the same, and streaks represent nothing more than the lucky atom that kept moving in one direction. Long hitting streaks happen to the greatest players—Sisler, Keeler, DiMaggio, Rose—because their general chance of getting a hit is so much higher than average. Just as Joe Airball cannot match Larry Bird for runs of baskets, Joe’s cousin Bill Ofer, with a lifetime batting average of .184, will never have a streak to match DiMaggio’s with a lifetime average of .325. The statistics show something else, and something fascinating: there is no “causality of circumstance,” no “extra” that the great can draw from the soul of their valor to extend a streak beyond the ordinary expectation of coin-tossing models for a series of unconnected events, each occurring with the characteristic probability for that particular player. Good players have higher characteristic probabilities, hence longer streaks.
Of course DiMaggio had a little luck during his streak. That’s what streaks are all about. No long sequence has ever been entirely sustained in any other way (the Orioles almost won several of those twenty-one games). DiMaggio’s remarkable achievement—its uniqueness, in the unvarnished literal sense of that word—lies in whatever he did to extend his success well beyond the reasonable expectations of random models that have governed every other streak or slump in the history of baseball.
Probability does pervade the universe—and in this sense, the old chestnut about baseball imitating life really has validity. The statistics of streaks and slumps, properly understood, do teach an important lesson about epistemology, and life in general. The history of a species, or any natural phenomenon that requires unbroken continuity in a world of trouble, works like a batting streak. All are games of a gambler playing with a limited stake against a house with infinite resources. The gambler must eventually go bust. His aim can only be to stick around as long as possible, to have some fun while he’s at it, and, if he happens to be a moral agent as well, to worry about staying the course with honor. The best of us will try to live by a few simple rules: do justly, love mercy, walk humbly with thy God, and never draw to an inside straight.
DiMaggio’s hitting streak is the finest of legitimate legends because it embodies the essence of the battle that truly defines our lives. DiMaggio activated the greatest and most unattainable dream of all humanity, the hope and chimera of all sages and shamans: he cheated death, at least for a whil