If you can explain anything, it proves nothing
An excellent piece from sports site Grantland (via Brendan Nyhan), on finding explanations for random noise and regression to the mean.
As a demonstration, they took ten baseball batters and ten pitchers who had apparently improved over the season so far, and searched the internet for news that would allow them to find an explanation. They got pretty good explanations for all twenty. Looking at past seasons, this sort of short-term improvement almost always turns out be random noise, despite the convincing stories.
Having a good explanation for a trend feels like convincing evidence the trend is real. It feels that way to statisticians as well, but it isn’t true.
It’s traditional at this point to come up with evolutionary psychology explanations for why people are so good at over-interpreting trends, but I hope the circularity of that approach is obvious.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
Hi Thomas, interesting piece. Similar to the point the book “the tiger that isn’t” makes.
If you can add some more examples, the Ddj community would appreciate
BR
Ben
10 years ago