Risk and blame

Ruins after the L'Aquila earthquake
http://i.telegraph.co.uk

by Ian Pollock

When six Italian scientists and civil servants were put on trial, then convicted on October 22, 2012 of multiple manslaughter (with a sentence of six years in prison each), the reaction from the scientific community was fast and for the most part negative. The defendants, who had been part of a public advisory board on safety risks, had been charged with inducing a false sense of security among the residents of L’Aquila (which had recently experienced strong tremors), causing many residents to stay at home, where approximately 300 were killed by a terrible earthquake in April 2009.

The issue is not as simple as the American Association for the Advancement of Science seemed to believe when it wrote a letter in June 2010 to the Italian President. In it, the AAAS claimed that the basis of the indictment was that the scientists failed to alert the population of L’Aquila to the impending disaster. This is not quite right — the charge was not of insufficient predictive ability, but of culpably poor risk communication. Although at a public meeting in March 2009, a large earthquake was called “unlikely,” but not excluded as a possibility, officer Bernardo de Bernardinis afterward assured the public that there was “no danger,” and that the tremors were good inasmuch as they released pent-up tectonic energy — agreeing with a reporter’s offhand suggestion that residents should crack open a bottle of wine.

This is not as misleading as it sounds, for although around half of all large quakes are preceded by minor foreshocks [P(tremor|large quake)], a minor earthquake is followed by another at least 1.0 magnitude unit larger only 1-3% of the time [P(large quake|tremor)] — which is the relevant statistic for the situation. Nonetheless, tremors are positive (if very weak) Bayesian evidence in favor of an impending larger quake: this does look like poor science and poor science communication. We do not know how many people died specifically because of the scientists’ downplaying of the risks, but it’s at least plausible that it’s a substantial fraction of the 300. (Although for some perspective, consider that no one has been charged in relation to poor building standards in the town, despite their much more proximate causal link to the deaths.)

Even granting the charge of poor science communication, the judgment of six years for manslaughter is obviously extreme, not to mention encouraging of perverse incentives. When a Type I error leads to some annoyed residents, and a Type II error leads to half a dozen years in jail, count on future risk experts to always cover their asses by overplaying every danger, no matter how unlikely. This, of course, leads to the public discounting experts, and then ignoring serious counsel they should heed. As Daniel Kahneman says in “Thinking Fast and Slow,” increased accountability is a mixed blessing. It leads to the extinction of useful signals from experts, and their replacement with weasel words.

There are lots of people in the world to feel much sorrier for, but I do often feel pretty sorry for the people in charge of communicating risk to the public and taking actions to mitigate it. In addition to the difficulty they have in predicting and preparing for the future (which obviously varies depending on what is being predicted), they face several huge cognitive problems with their audience:

• Neglect of probability: “Don’t give me the odds! Is it safe or not?”
• Denial of personal responsibility: “Nobody informed us of the risks of waterfall kayaking.”
• Bad-guy bias: “My son died on the operating table, and someone needs to be held responsible!”
• Hindsight/Outcome bias and moral luck: “How can you say it was the right call based on what you knew, when forty families are grieving!”
• Moral grandstanding: “No risk to our children is acceptable!”
• Crying-probable-wolf effect: “The last two evacuations were false alarms. I’m not going anywhere.”

Once you are made aware of these phenomena, you start noticing the pattern all over the place.

Kahneman relates the following anecdote:

“The worse the consequence, the greater the hindsight bias. In the case of a catastrophe, such as 9/11, we are especially ready to believe that the officials who failed to anticipate it were negligent or blind. On July 10, 2001, the Central Intelligence Agency obtained information that al-Qaeda might be planning a major attack against the United States. George Tenet, director of the CIA, brought the information not to President George W. Bush but to National Security Adviser Condoleezza Rice. When the facts later emerged, Ben Bradlee, the legendary executive editor of The Washington Post, declared, ‘It seems to me elementary that if you’ve got the story that’s going to dominate history you might as well go right to the president.’ But on July 10, no one knew — or could have known — that this tidbit of intelligence would turn out to dominate history.”

This would almost be funny if it weren’t so malicious, as here the usual blind spots are amplified by political point-scoring.

The biases affecting risk assessment and the assignation of blame for it are now well-publicized, for example by Kahneman, as well as by Dan Gardner in his excellent book “Risk.” However, we have to take this knowledge out of the realm of books on cognitive psychology and apply it wisely in real world cases. It is important to note that being made aware of the possibility of, for example, hindsight bias only reduces it very slightly. This makes me wonder if even my fairly exculpating response to the earthquake case is still too hawkish. Had no quake occurred, would I have judged de Bernardinis’ misleading statement as anything more than a minor slip of the tongue? (Other commentators fell more spectacularly for hindsight, pointing to the disastrous quake of 1703 as evidence that the group should obviously have known better.) 

I think the lesson from this is that we ought to cut the people in charge of controlling public risks, whether they are employed in the public or private sector, a great deal more slack. Our prior for culpable negligence on their part is always going to be overinflated in the aftermath of a disaster that actually did happen. We should be especially slow to judge when we or someone we know has been affected, or when our political views already predispose us against the decision makers.

One of the more powerful results of decision theory is that pretty much all decisions can be modeled as gambles. Many of those gambles have stakes that are measured in human lives — including banal decisions such as whether to go to work while you have the flu. At the level of public policy, the stakes are necessarily much higher, the uncertainty is usually greater, and alas, by its nature the gamble cannot simply be avoided.

Not only that, but just like private citizens, decision makers dealing with risk have to consider other factors besides risk to human life — such as “mere” convenience or “mere” money. Although private citizens are more than happy to grandstand about the infinite value of human life (and then go out driving in icy weather because they really feel like a cheap donair), policy decision makers are obliged to explicitly consider e.g., the economic impact of an evacuation against its potential to save lives. These are agonizing but necessary choices.

So let us think once, twice, and three more times before we demand somebody’s scalp because something really bad happened. Especially if it “could have been prevented.”