Newcomb’s Paradox: An Argument for Irrationality

By Julia Galef

I had heard rumors that Newcomb’s Paradox was fiendishly difficult, so when I read it I was surprised at how easy it seemed. Here’s the setup: You’re presented with two boxes, one open and one closed. In the open one, you can see a $1000 bill. But what’s in the closed one? Well, either nothing, or $1 million. And here are your choices: you may either take both boxes, or just the closed box.

But before you think “Gee, she wasn't kidding, that really is easy,” let me finish: these boxes were prepared by a computer program which, employing advanced predictive algorithms, is able to analyze all the nuances of your character and past behavior and predict your choice with near-perfect accuracy. And if the computer predicted that you would choose to take just the closed box, then it has put $1 million in it; if the computer predicted you would take both boxes, then it has put nothing in the closed box.

So, okay, a bit more complicated now, but still an obvious choice, right? I described the problem to my best friend and said I thought the question of whether to take one box or both boxes was pretty obvious. He agreed, “Yeah, this is a really easy problem!”

Turns out, however, that we each were thinking of opposite solutions as the “obviously” correct one (I was a two-boxer, he was a one-boxer.) And it also turns out we’re not atypical. Robert Nozick, the philosopher who introduced this problem to the public, later remarked, “To almost everyone it is perfectly clear and obvious what should be done. The difficulty is that people seem to divide almost evenly on the problem, with large numbers thinking that the opposing half is just being silly."

Anyway, thence began my trajectory of trying to grok this vexing problem. It's been a wildly swinging trajectory, which I'll trace out briefly and then explain why I think Newcomb's Paradox is more relevant to real life than your typical parlor game of a brain-teaser.

So as I mentioned, first I was a two-boxer, for the simple reason that the closed box is sealed now and its contents can't be changed. Regardless of what the computer predicted I'd do, it's a done deal, and either way I'm better off taking both boxes. Those silly one-boxers must be succumbing to some kind of magical thinking, I figured, imagining that their decision now can affect the outcome of a decision that happened in the past.

But then of course, I had to acknowledge that nearly all the people who followed the two-boxing strategy would end up worse off than the one-boxers, because the computer is stipulated to be a near-perfect predictor. The expected value of taking one box is far greater than the expected value of taking both boxes. And so the problem seems to throw up a contradiction between two equally intuitive definitions of rational decision-making: (1) take the action with the greater expected value outcome, i.e., one-box; versus (2) take the action which, conditional on the current state of the world, guarantees you a better outcome than any other action, i.e., two-box.

So then I started leaning toward the idea that this contradiction must be a sign that there was some logical impossibility in the setup of the problem. But if there is, I can't figure out what. It certainly seems possible in principle, even if not yet in practice, for an algorithm to predict someone's future behavior with high accuracy, given enough data from the past (and given that we live in a roughly deterministic universe).

Finally, I came to the following conclusion: before the box is sealed, the most rational approach is (1), and you should intend to one-box. After the box is sealed your best approach is (2) and you should be a two-boxer. Unfortunately, because the computer is such a good predictor, you can't intend to be a one-boxer and then switch to two-boxing, or the computer will have anticipated that already. So your only hope is to find some way to pre-commit to one-boxing before the machine seals the box, to execute some kind of mental jujitsu move on yourself so that your rational instincts shut off once that box is sealed. And indeed, according to my friends who study economics and decision theory, this is a commonly accepted answer, though there is no really solid consensus on the problem yet.

Now here's the real-life analogy I promised you (adapted from Gary Drescher's thought-provoking Good and Real): imagine you're stranded on a desert island, dying of hunger and thirst. A man in a rowboat happens to paddle by, and offers to transport you back to shore, if you promise to give him $1000 once you get there. But take heed: this man is extremely psychologically astute, and if you lie to him, he'll almost certainly be able to read it in your face.

So you see where I'm going with this: you'll be far better off if you can promise him the money, and sincerely mean it, because that way you get to live. But if you're rational, you can't make that promise sincerely — because you know that once he takes you to shore, your most rational move at that stage will be to say, “Sorry, sucka!” and head off with both your life and your money. If only you could somehow pre-commit now to being irrational later!

Okay, so maybe in your life you don't often find yourself marooned on a desert island facing a psychologically shrewd fisherman. But somewhat less stylized versions of this situation do occur frequently in the real world, wherein people must decide whether to help a stranger and trust that he will, for no rational reason, repay them.

Luckily, in real life, we do have a built in mechanism that allows us – even forces us – to pre-commit to irrational decision-making. In fact, we have at least a couple such mechanisms: guilt and gratitude. Thanks to the way evolution and society seem to have wired our brains, we know we'll feel grateful to people who help us and want to reward them later, even if we could get off scot-free without doing so; or at the least we'll suffer from feelings of guilt if we break our promise and screw them over after they've helped us. And as long as we know that -- and the strangers know we know that -- they're willing to help us, and as a result we end up far better off.

Thus dies Newcomb's Paradox, at least the real-world version of it: slain by rational irrationality.