About Rationally Speaking


Rationally Speaking is a blog maintained by Prof. Massimo Pigliucci, a philosopher at the City University of New York. The blog reflects the Enlightenment figure Marquis de Condorcet's idea of what a public intellectual (yes, we know, that's such a bad word) ought to be: someone who devotes himself to "the tracking down of prejudices in the hiding places where priests, the schools, the government, and all long-established institutions had gathered and protected them." You're welcome. Please notice that the contents of this blog can be reprinted under the standard Creative Commons license.

Thursday, September 29, 2005

Are crowds wise?

I'm intrigued, a bit puzzled, and a little annoyed by James Surowiecki's book, The Wisdom of Crowds. The book explores the many facets and consequences of a phenomenon noted first by Francis Galton (Darwin's cousin, a brilliant statistician, but also the founder of the infamous movement of eugenics). Galton went to a Fat Stock and Poultry exhibition and came across a weight-judging contest. The crowd was invited to guess the weight of a fat ox, buying a ticket per guess, with a large pot of money as the prize for the person who got closest.

Almost eight hundred people tried, many of whom were not experts at judging the weight of large mammals. Galton collected the data, wishing to test the hypothesis that the average would be significantly off the mark, and that a few people -- presumably with more expertise than the average -- would instead get very close to the real number. He published the results in the journal Nature, where he noted his surprise at the fact that the weight of the ox, once slaughtered and dressed (that was the bet), was 1,197 pounds. The actual result was 1,198, an error rate of only 0.0008! Galton's comment was that "the result seems more creditable to the trustworthiness of a democratic judgment than might have been expected."

As Surowiecki notes, this sort of phenomenon -- to which he refers to as the "wisdom" of the crowds -- has been repeatedly documented since, and the book presents several intriguing examples. The thing is a bit puzzling, until one realizes that it works under certain sets of circumstances and for certain types of problems.

The conditions include: diversity of opinion (obviously), independence (i.e., people are not communicating and/or arguing with each other), decentralization (people can draw on local knowledge, including the possibility of specialization), and aggregation (there has to be a mechanism to turn independent private judgments into a collective decision).

This works because of what really is a simple truth in mathematical statistics: the errors made by each independent individual will be of similar magnitude but different direction and, if the group is large enough, on average they will cancel each other, so that the group's estimate will turn out to be remarkably accurate.

Of course, this works for certain categories of problems only, though Surowiecki attempts to show that these categories are so broad that the method should be used to solve all sorts of human problems (which is, of course, a gross exaggeration). The kinds of problems where crowds might be "wiser" than most individuals include: cognition problems (such as the problem of the ox mentioned above), coordination problems (where coordination with other members of a group is required, as in walking a crowded street), and cooperation problems (where one wants self-interested people to cooperate, as in paying taxes). While these categories seem indeed almost all-encompassing, Surowiecki himself gives plenty of examples where the crowds don't perform better than the experts, because often one or more of the conditions given above are violated (for example, faculty meetings tend to be particularly inefficient, because they are small groups, opinions are not independent, and they tend not to be sufficiently diverse). Moreover, it isn't really all that clear that the "wisdom" involved in the three cases is really the same sort of phenomenon (after all, the three kinds of tasks are very different, and presumably they depend on different cognitive abilities).

This makes for very interesting reading, but there are two major caveats that need to be kept in mind. First, none of the three categories of problems identified by Surowiecki has anything to do with "wisdom." Which undermines one of the major messages of the book, that democracies are a "wise" form of government because they are the expression of the will of a large number of people. Second, there is a more subtle, almost anti-intellectual, message here, that "experts" aren't to be trusted, because they don't do as well as crowds. This populist attitude feeds into the already generally anti-intellectual American public, and may have serious consequences when it comes to addressing problems that don't fit the classification or meet the requirements laid out by Surowiecki. Which, I suspect, happens a lot more often than the author would like to admit.

5 comments:

  1. Which crowd?

    http://towardtradition.org/article
    _God_problems.htm

    "Once upon a time in St Louis, there lived a husband who yearned to vacation in Los Angeles while his wife craved a few days in New York. Averaging their desires and finding the midpoint, the couple went to Kansas City. Averaging doesn’t always work.

    Averaging doesn’t work when you are dealing with two very different things like Los Angeles and New York or like traditional Americans and those of the secular fundamentalist persuasion..."

    ReplyDelete
  2. I haven't read the book, but from the descriptions of it I've heard the "wisdom" of the crowds is just a neat averaging phenomenon - 30 people are better at guesstimating that 5.

    But no crowd is going to guesstimate Planck's constant.

    And perhaps people should read Extraordinary Popular Delusions & the Madness of Crowds in conjunction with this book.

    ReplyDelete
  3. Massimo, can you give the full citation to Galton's paper in Nature? Thanks.

    ReplyDelete
  4. The full ref to Galton's paper is:

    Francis Galton, "Vox Populi," Nature vol. 75, pp. 450-451, March 7, 1907.

    ReplyDelete
  5. Anonymous up there has good points: "which crowd?" "avrg does not always work".

    Sure, it has to be well done, following well stablished statistical practices and all that boring stuff, blah, blah.

    But the link provided by "anon" is simply ridiculous, or is it just me? The rabi there starts well, criticizing some study that was published in an English newspaper, I think. I don't know if his critiques are accurate or not, but they do sound reasonable, believable and all that.

    Then, to the (brain equipped) reader, comes the surprise: the rabi's thesis is exactly the opposite (no surprise HERE), but he does not even try to keep up to the standards he demanded from the other guys. All he presents are his prejudices. No data, no analysis, no reason at all for his stupid conclusion. He does not even try to explain, eg., how on Earth can the 10% or so "secular America" (is that the number) be responsible for a 50% divorce rate (which he says does not happen in "his" America), or some other "horrible" statistic of your choice. Which means he revealed himself to be, in his own words, nothing but "an ideologue with a political agenda instead of a thesis".

    And then these people get angry when they are ridiculed. Oh, well, what can you expect...

    Cheers
    J

    ReplyDelete

Note: Only a member of this blog may post a comment.