The probability of Gremlins

I’m reading philosopher Elliott Sober’s Evidence and Evolution: The Logic Behind the Science (I am in the midst of reviewing the book for Trends in Ecology & Evolution), and the first chapter presents a nice little example about probabilities and likelihoods that is pertinent to anyone interested in hypothesis testing and pseudoscience.

The context of Sober’s discussion is a chapter devoted to inference, and in particular to the advantages and disadvantages of Bayesian vs. frequentist statistical frameworks. There is a crucial and often unappreciated distinction between the probability of a given hypothesis being true if one observes certain data [ P(HD) in Bayesian terms] vs. the probability of observing certain data if a given hypothesis is true [ P(DH) ]. Here is how Sober explains it (p. 10 of the book):

“Suppose you hear a noise coming from the attic of your house. The likelihood of this hypothesis [ P(DH) ] is very high, since if there are gremlins bowling in the attic, there probably will be noise. But surely you don’t think that the noise makes it very probable that there are gremlins up there bowling,” because that probability of that hypothesis given the data, P(HD), is very low indeed.

Notice that P(DH) is referred to as the likelihood (of the data given the hypothesis), while P(HD) is the (posterior) probability of the hypothesis given the data. A confusion between these two quantities underlies much pseudoscience. Sober’s own example is not far fetched at all: people who believe in ghosts and haunted houses use exactly that sort of “reasoning.” Believers in UFOs also fall into the same trap: if you observe strange lights in the night sky, the likelihood that those are due to an extraterrestrial spaceship is high if there is a good a priori chance (a high prior in Bayesian jargon) that we really are visited by space aliens. But the probability of the UFO hypothesis being true just on the basis that you observed unidentified lights in night is very, very low.

An analogous application of the distinction between likelihoods and posterior probabilities can be made in the case of intelligent design “theory.” The observation of a complex biological structure such as the bacterial flagellum has a high likelihood if one assumes that there is a supernatural intelligent designer messing around with the universe. But the probability that there is an intelligent designer simply based on the fact that we observe complex biological structures is, again, vanishingly small.

If you’d like to play with an online Bayesian calculator, check this one here for simple situations (two hypotheses), or this one here for more complex ones (up to five competitive hypotheses). Just remember, don’t confuse your likelihoods with your probabilities!