Tonight’s Mythbusters episode on Prime looked at the Monty Hall/Pick-a-Door problem, using experimental data as well as theory.
For those of you who haven’t been exposed to it, the idea is as follows:
There are three doors. Behind one is a prize. The contestant picks a door. The host then always opens one of the other doors, which he knows does not contain the prize. The contestant is given an opportunity to change their choice to the other unopened door. Should they take this choice?
The stipulation that the host always makes the offer and always opens an empty door is critical to the analysis. It was present in the original game-show problem and was explicit in Mythbusters.
A probabilistic analysis is straightforward. The chance that the prize is behind the originally-chosen door is 1/3. It has to be somewhere. So the chance of it being behind the remaining door is 2/3. You can do this more carefully by enumerating all possibilities, and you get the same answer.
The conclusion is surprising. Almost everyone, famously including
both Marilyn vos Savant, and Paul Erdős, gets it wrong. Less impressively, so did I as an undergraduate, until I was convinced by writing a computer simulation (I didn’t need to run it; writing it was enough). The compelling error is probably an example of the endowment effect.
All of the Mythbusters live subjects chose to keep their original choice,ruining the comparison. The Mythbusters then ran a moderately large series of random choices where one person always switched and the other did not. They got 38 wins out of 49 for switching and 11 for not switching. That’s a bit more extreme than you’d expect, but not unreasonably so. It gives a 95% confidence interval (analogous to the polling margin of error) from 12% to 37%.
The Mythbusters are sometimes criticised for insufficient replication, but in this case 49 is plenty to distinguish the ‘obvious’ 50% success rate from the true 33%. It was a very nicely designed experiment.