March 3, 2013

Keep calm and ignore tail risk

Consider a problem in statistical decision theory.

Suppose you can create a t-shirt slogan at zero cost, and submit it for market testing.  If it’s popular, you make money; if it isn’t popular, you pay nothing.  It’s easy to see that you should submit as many t-shirt designs as you can generate: there’s no downside, and the upside might be good.

The problem is that you might create slogans that are sufficiently offensive to get the whole world mad at you.  And if you create more than half a million t-shirt slogans, it’s not all that unlikely that some of them will be really, really, really bad. And it’s not a convincing defense to say that the computer did it, and you didn’t bother checking the results.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »