Stay alert for overselling.
From the Daily Mail (via the Herald) “Need a boost? Try orange juice, not coffee“, reports on a study comparing high-pulp orange juice not to coffee but to orange-flavoured water. The story says
After the real juice they did better on tests of speed and attention and still felt very alert six hours later, the European Journal of Nutrition reports.
The research paper is here (open access, no link). There were ten tests of speed and attention and mood, each done at two times after the orange juice. If you chose the most-impressive of the twenty comparisons and pretended it was the only one that mattered, you’d get some reasonable evidence that orange juice gave a slight improvement over fake orange juice. If you take into account the changes in all the measurements it looks much less convincing. A combined analysis of all the measurements “approached significance,” as people say when they don’t get the hoped-for results.
And “felt very alert six hours later“? That’s a 6% difference between fake and real orange juice on a “how alert do you feel?” scale, plus or minus about 6.4%.
It’s a pity that Pepsi, who sponsored the study and sell the juice, didn’t make it a bit bigger so that any real effects would be convincing and chance fundings would be more clearly too small to worry about.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »