Sir Francis Galton was a celebrated British scientist. See, http://en.wikipedia.org/wiki/Francis_Galton He found that farmers, butchers, and non-experts could as a voting group accurately guess the weight of an ox. http://galton.org/essays/1900-1911/galton-1907-vox-populi.pdf "In these democratic days, any investigation into the trustworthiness and peculiarities of popular judgments is of interest. The material about to be discussed refers to a small matter, but is much to the point. A weight-judging competition was carried on at the annual show of the West of England Fat Stock and Poultry Exhibition recently held at Plymouth, A fat ox having been selected, competitors bought stamped and numbered cards, for 6d. each, on which to inscribe their respective names, addresses, and estimates of what the ox would weigh after it had been slaughtered and " dressed." Those who guessed most successfully received prizes. About 800 tickets were issued, which were kindly lent me for examination after they had fulfilled their immediate purpose. These afforded excellent material. The judgments were unbiased by passion and uninfluenced by oratory and the like. The sixpenny fee deterred practical joking, and the hope of a prize and the joy of competition prompted each competitor to do his best. The competitors included butchers and farmers, some of whom were highly expert in judging the weight of cattle; others were probably guided by such information as they might pick up, and by their own fancies" "The average competitor was probably as well fitted for making a just estimate of the dressed weight of the ox, as an average voter is of judging the merits of most political issues on which he votes, and the variety among the voters to judge justly was probably much the same in either case. After weeding thirteen cards out of the collection, as being defective or illegible, there remained 787 for discussion. I arrayed them in order of the magnitudes of the estimates, and converted the cwt., quarters, and lbs, in which they were made, into lbs., under which form they will be treated." "According to the democratic principle of "one vote one value," the middlemost estimate expresses the vox populi, every other estimate being condemned as too low or too high by a majority of the voters (for fuller explanation see, One Vote, One Value," Nature, February 28, p. 414). Now the middlemost estimate is 1207 lb., and the weight of the dressed ox proved to be 1198 lb.; so the vox populi was in this case 9 lb., or 0.8 per cent of the whole weight too high."Galton continues with these conclusions: "It appears then, in this particular instance, that the vox populi is correct to within 1 per cent of the real value, and that the individual estimates are abnormally distributed in such a way that it is an equal chance whether one of them, selected at random, falls within or without the limits of -3.7 per cent and +2.4 per cent of their middlemost value.
The authorities of the more important cattle shows might do service to statistics if they made a practice of preserving the sets of cards of this description, that they may obtain on future occasions, and loaned them under proper restrictions, as those have been, for statistical discussion. The fact of the cards being numbered makes it possible to ascertain whether any given set is complete." ## Discussion On Galton's ConclusionsGalton's cattle weighing study is cited by many researchers TBD so it is worthwhile to consider them. Galton says he had access to 787 weight-judging competition tickets. He provides the raw data in the image above from his vox populi article however I do not see any numbers amounting to 787. Galton labels his data "Distribution of the estimates" but does not seem to provide the full underlying data. Galton picks out on a post-hoc basis the http://galton.org/essays/1900-1911/galton-1907-vote-value.pdf "How can the right conclusion be reached, considering that there may be as many different estimates as there may are members? The conclusion is clearly not the average of all estimates which would give a voting poser to "cranks" in proportion to their crankiness. ... I wish to point out that the estimate to which is least objection can be raised is the middlemost estimate." In my opinion, the weighted median value is probably the best estimate - not Galton's median value. ## Additional Problems With Galton's Conclusions- Many people who write about the wisdom of the crowds cite Galton's 1907 ox-weighing research. Where are the replicated studies? There should be at least several research articles verifying what Galton purported to show but I don't think these studies exist. Plenty of state fairs have cattle weighing contests so there should be many more studies.
- Galton says that farmers and ranchers and the average non-farmer who visited the West of England Fat Stock and Poultry Exhibition were all about equally expert in judging the weight of cattle. Farmers weigh their livestock with scales and tape measures and they also go to livestock auctions. There they gain a lot of expertise on visually estimating the weight of cattle.
- Galton equates guessing the weight of an ox with the ability to assess the qualities of a politician. That seems to be an intellectual stretch.
- Cattle have ascertainable weights - put them on a scale and you can get a scientific metric such as: This ox weighs 1207 pounds. Politicians have a lot of undefinable characteristics.
## My ConclusionsGalton's research is faulty for several reasons including: - His findings need to be replicated.
- His use of the median value appears to be post-hoc and he doesn't discuss why he didn't choose the weighted median which to me seems a better metric.
- His relating ox-weight guessing to democratic voting needs support.
## Critical Review of the Best-Seller: The Wisdom of Crowds, by James Surowiecki
This book is widely praised, for example:
I do not have this book, but some material is available as a preview on Amazon here: http://www.amazon.com/Wisdom-Crowds-James-Surowiecki/dp/0385721706#reader_0385721706 This is a quote from James Surowiecki's introduction that in my opinion has some serious "Galton
was interested in figuring out what the "average voter" was capable of
because he wanted to prove that the average voter was capable of very
little. So he turned the competition into an impromptu experiment. When
the contest was over and the prizes had been awarded, Galton borrowed
the tickets from the organizers and ran a series of statistical tests on
them. Galton arranged the guesses (which totaled 787 in all after he
had to discard thirteen because they were illegible) in order from
highest to lowest and graphed them to see if they would form a bell
curve. Then among other things, he added all of the contestants'
estimates, and calculated the mean of the group's guesses. That number
represented, you could say, the collective wisdom of the Plymouth crowd.
If the crowd were a single person, that was how much it would have
guessed the ox weighed. ## Bugs
However, Galton did Galton: "According to the democratic principle of "one vote one value," the middlemost estimate expresses the vox populi, every other estimate being condemned as too low or too high by a majority of the voters (for fuller explanation see, One Vote, One Value," Nature, February 28, p. 414). In fact, in a different earlier article by Galton, he condemns using the average (the mean): http://galton.org/essays/1900-1911/galton-1907-vote-value.pdf "How
can the right conclusion be reached, considering that there may be as
many different estimates as there may are members? ."middlemost estimate
The Take
a look at the data in the chart above then add up the second column
marked "Estimates in lbs." (22,747 total) then divide that total by the
number of entries (19) in the column. The result is The
19 entries in Galton's data represent the various ox-weight estimates.
Galton does not appear to have provided information on how many people
guessed each of the 19 estimate values. Therefore, the To
provide the crowd's average guess we would need to add the 787
ox-weight guesses then divide that sum by 787, which Galton did not
report. ## ReferencesAnother animal weight-guessing competition: Auckland Star, Volume XXX, Issue 279, 24 November 1899, Page 8. An old study from "Nearly all competitors over-estimated the weight of the sheep, and under-estimated the weight of the cattle." How would Galton in 1907 have interpreted the sheep and cattle weight guessing differences? |

Ensemble Classifiers > Group Intelligence >