Posts written by Thomas Lumley (1983)


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

April 10, 2017


  • Good piece at Stuff about what a 500-year flood is. The concept isn’t quite as shaky as it sounds — there’s some independent information from comparing different river systems — but it’s inevitably uncertain.
  • 23andme is back providing genetic risk information, but in a much more restricted way after FDA review.  A lot of the risk information you can get this way isn’t useful for treatment, but it’s the sort of thing some people like to know.  So, sometimes, do their insurance companies
  • The concept of ‘net tax’ — tax paid minus cash benefits and transfers (but not non-cash ones such as Pharmac subsidies) can be a useful concept.  However, I don’t think it’s as useful when ‘tax’ leaves out GST, as in this story at Stuff.  Admittedly, it’s not trivial to calculate how much GST people pay, but I’m sure the Treasury had looked at it.
  • Scientists and journalists need to get better at communicating uncertainty, and people need to accept it’s there. (Ed Yong, in the Atlantic)
April 5, 2017

Extrapolation, much?

HeadlineResearch has found that Marmite could help prevent dementia

Research article:  A group of 28 adult volunteers (10 males, mean age 22 years) completed the study after providing written informed consent.

We could just stop there, but it gets better (not better)

The study found that the people getting Marmite had, as hypothesised, less response by their brains to flickering visual stimuli.  The research paper does not mention dementia (or memory, or Alzheimers). At all. It concludes

“This demonstrates that the balance of excitation and inhibition in the brain can be influenced by dietary interventions, suggesting possible clinical benefits in conditions (e.g. epilepsy) where inhibition is abnormal.”

Even the story doesn’t come close to the headline claims, saying just

It could also prompt further research to see if Marmite, and its effect on the brain’s GABA chemical, might provide a treatment for dementia.

And, right at the end of the story, the quote from an independent expert

“there’s no way to say from this study whether eating Marmite does affect your dementia risk.

If it does, and if that’s because of the vitamin B12, it might also have been worth mentioning that there are other foods with as much or more vitamin B12 per serving, such as beef, and lamb, and many types of fish.



  • If someone told me a longstanding problem in mathematical statistics had been solved, but then admitted the proof was short, used fairly elementary techniques, was written with Microsoft Word, and was published in the Far East Journal of Theoretical Statistics, I might not be in a hurry to look it up.  These are all genuinely reasonable filters for mathematical papers that are worth putting effort into. But, in this case, they were all false positives. Quanta Magazine has the story.
  • From The Conversation,”The seven deadly sins of statistical misinterpretation, and how to avoid them“.
  • From Newsroom (who seem to be quite good so far) Interaction of recreational genotyping and health insurance in NZ
  • From The Conversation, how website terms of use (and their potential criminal enforcement in the US) affect research into fairness and transparency of algorithms.
  • Good Herald interview on air pollution with NIWA scientist Elizabeth Somervell
April 4, 2017

Attack of the killer margarine: the reboot

In 2015, the Herald had a story from the Daily Telegraph on the alleged risks of margarine:

Saturated fat found in butter, meat or cream is unlikely to kill you, but margarine just might, new research suggests.

Traditionally people have been advised to reduce animal fats, but the biggest ever study has shown they do not increase the risk of stroke, heart disease or diabetes. However, trans fats, found in processed foods such as margarine, raise the risk of death by 34 per cent in less than a decade.

“For years everyone has been advised to cut out fats,” said study lead author Doctor Russell de Souza, an assistant professor in the Department of Clinical Epidemiology and Biostatistics, at McMaster University in Canada.

It’s a bit unclear exactly what “raise the risk of death by 34 per cent in less than a decade” is supposed to mean, but we’ll get to that. The research paper was in the BMJ, and came out on the same day the story did.

Today, in 2017, the Herald had a story from the Daily Telegraph on the alleged risks of margarine:

Saturated fat found in butter, meat or cream is unlikely to kill you, but margarine just might, new research suggests.

Although traditionally dieticians have advised people to cut down on animal fats, the biggest ever study has shown that it does not increase the risk of stroke, heart disease or diabetes.

However trans-fats, found in processed foods like margarine raises the risk of death by 34 per cent.

“For years everyone has been advised to cut out fats,” said study lead author Doctor Russell de Souza, an assistant professor in the Department of Clinical Epidemiology and Biostatistics, at McMaster University in Canada.

It’s a bit unclear exactly what “raise the risk of death by 34 per cent in less than a decade” is supposed to mean, but we’ll get to that. The research paper was in the BMJ, and came out nearly two years before the story did.

Yes, it really seems to be the same ‘new reasearch’: Dr de Souza hasn’t just published another meta-analysis. It even seems to be the same Telegraph story; I couldn’t find a new one.

So, how scared should we be of trans fats in our diets?  Food Standards Australia New Zealand say

Monitoring of TFAs in the Australian and New Zealand food supply has found that Australians obtain on average 0.5 per cent of their daily energy intake from TFAs and New Zealanders on average 0.6 per cent. This is well below the WHO recommendation of no more than 1 per cent.

They also say that the majority of that 0.6% is made by bacteria in the rumens of cows and sheep, not by industrial hydrogenation; the evidence of harm is weaker for these natural trans fats.

Now, back to the 34% statistic. This is based on two studies. One compared the 20% of people with the highest and lowest trans fat intakes and found a rate ratio of 1.24. The other, smaller, one estimated the ratio as 1.71 between the highest and lowest 25%.   These are rate ratios estimated from people in their 60. Since the actual probability of death in any given year would have been about 1% the absolute risk increase is smaller than “34% in less than a decade” sounds — but not at all trivial.  For comparison, the all-cause mortality rate ratio for current smoking is about 3.0, or 200% higher than non-smokers.

More importantly, though, we’re talking about a lot of trans fat in these studies. In the larger study with the less-scary rate ratio, people in the lowest 20% of trans fat intake got an average of 1.6% of their calories from it. That is, the lowest-risk group were eating three times as much trans fat as an average Kiwi today.  In the smaller study, they don’t give actual trans fat information for the groups they are comparing, but the average for the whole study was about 9% of fat in the blood was trans fat: if that even roughly translates to proportions of dietary fat they were also getting more than the typical Kiwi today.

There just isn’t that much trans fat in most margarine any more, less than 1% on average (according to Food Standards Oz/NZ, table 2) . There used to be a lot, but then we found out it’s bad for you.  Those scary numbers are actually good news if they’re true: they’d measure how much better off margarine consumers are today than twenty years ago.

(via Mark Hanna)

How big is that (2)

In yesterday’s Official StatsChat Bogus Poll, about two-thirds of the respondents got one of the reasonable answers.

Here’s how we could work out most of the answer without looking it up.

First, a hectare is 10,000 square meters, or 2.5 acres (you might need to look that up).

Now, a ‘full section’ for a house is typically less than 1000 square meters (a quarter acre), so you get 10-20 of them per hectare, and maybe 100,000 of them in 7000 ha.  That’s definitely bigger than Eden Park, and it’s pretty clearly bigger than One Tree Hill Domain + Cornwall Park.  In the other direction, 100,000 full sections must be smaller than the Auckland isthmus, and so (look at a map) smaller than Manukau Harbour.

The comparison to Epsom electorate is a little harder, and you might need actual data to decide.

Now, actual data:

The areas are

Eden Park:  originally `about 15 acres’, or 6 ha

One Tree Hill Domain + Cornwall Park: 270 ha

Epsom: 20 sq km, so 2,000 ha

Manukau Harbour: 394 sq km, or 39,400 ha.

Waikato Region: 2.5 million ha.

If you look up the area of  the Auckland isthmus for comparison with Manukau Harbour, you’ll probably get a figure of 638 sq km. That’s the area of the old Auckland City: it includes the Gulf Islands that were part of the city (in particular, Great Barrier and Waiheke make up more than half of it). It’s surprisingly hard to find the area of the isthmus itself on the internet.

April 3, 2017

How big is that?

From Stuff and the Science Media Centre

Dr Sean Weaver’s start-up business has saved over 7000 hectares of native rainforest in Southland and the Pacific

So, how much is that? I wasn’t sure, either.  Here’s an official StatsChat Bogus Poll to see how good your spatial numeracy is;

The recently ex-kids are ok

The New York Times had a story last week with the headline “Do Millennial Men Want Stay-at-Home Wives?”, and this depressing graphnyt

But, the graph doesn’t have any uncertainty indications, and while the General Social Survey is well-designed, that’s a pretty small age group (and also, an idiosyncratic definition of ‘millennial’)

So, I looked up the data and drew a graph with confidence intervals (full code here)


See the last point? The 2016 data have recently been released. Adding a year of data and uncertainty indications makes it clear there’s less support for the conclusion that it looked.

Other people did similar things: Emily Beam has a long post  including some context

The Pepin and Cotter piece, in fact, presents two additional figures in direct contrast with the garbage millennial theory – in Monitoring the Future, millennial men’s support for women in the public sphere has plateaued, not fallen; and attitudes about women working have continued to improve, not worsen. Their conclusion is, therefore, that they find some evidence of a move away from gender equality – a nuance that’s since been lost in the discussion of their work.

and Kieran Healy tweeted


As a rule if you see survey data (especially on a small subset of the population) without any uncertainty displayed, be suspicious.

Also, it’s impressive how easy these sorts of analysis are with modern technology. They used to require serious computing, expensive software, and potentially some work to access the data.  I did mine in an airport: commodity laptop, free WiFi, free software, user-friendly open-data archive.   One reason that basic statistics training has become much more useful in the past few decades is that so many of the other barriers to DIY analysis have been removed.

March 29, 2017

Technological progress in NZ polling

From a long story at

For the first time ever, Newshub and Reid Research will conduct 25 percent of its polling via the internet. The remaining 75 percent of polling will continue to be collected via landline phone calls, with its sampling size of 1000 respondents and its margin of error of 3.1 percent remaining unchanged. The addition of internet polling—aided by Trace Research and its director Andrew Zhu—will aim to enhance access to 18-35-year-olds, as well as better reflect the declining use of landlines in New Zealand.

This is probably a good thing, not just because it’s getting harder to sample people. Relying on landlines leads people who don’t understand polling to assume that, say, the Greens will do much better in the election than in the polls because their voters are younger. And they don’t.

The downside of polling over the internet is it’s much harder to tell from outside if someone is doing a reasonable job of it. From the position of a Newshub viewer, it may be hard even to distinguish bogus online clicky polls from serious internet-based opinion research. So it’s important that Trace Research gets this right, and that Newshub is careful about describing different sorts of internet surveys.

As Patrick Gower says in the story

“The interpretation of data by the media is crucial. You can have this methodology that we’re using and have it be bang on and perfect, but I could be too loose with the way I analyse and present that data, and all that hard work can be undone by that. So in the end, it comes down to me and the other people who present it.”

It does. And it’s encouraging to see that stated explicitly.

March 27, 2017

Nice to know it’s not men’s fault

Q: Did you see the headline? “Science proves beautiful women do make men cheat on their partners”?

A: Yes.

Q: Isn’t that a bit unethical? Doing experiments to make men cheat on their partners?

A: That’s not what they did.

Q: Well, not “make”, I suppose. “Encourage”.

A: “Seduce”, perhaps?

Q: Yes!

A: ◔_◔.  No.

Q: No what?

A: No actual cheating on partners was measured.  The study participants ‘cheated’ with small amounts of money — a few dollars.

Q: Oh. Did the researchers say this would extend to cheating on partners?

A: In the research paper they say “sexy women can seduce men away from behaving honestly and may more readily attract males who lie than was previously thought” and “For men whose mating motivation is heightened by exposure to sexual stimuli, dishonesty appears to be a tactic for projecting characteristics preferred by women (e.g., large economic resources)

Q: So beautiful women make men cheat to get small amounts of money, because then they’ll look rich and desirable.

A: That seems to be the theory.

Q: And this is the women’s fault.

A: They don’t quite come out and say that. But it makes men lose self-control

Q: Yes, this evidence that sexualised pictures make men lose self-control. How did they measure that?

A: The men took longer to answer computer questions involving the Stroop Effect: like “what colour is the word RED

Q: Is it just me who finds this slightly anticlimactic in context?

A: No.

Q: Isn’t the whole thing probably just chance?

A: They did have a preregistered replication of their main result, so that’s less likely than usual.

Q: How about that participants thought experimenters showing sexualised pictures of women were jerks and deserved to be taken for small sums of money?

A: Conceivable, I suppose. But that’s not an evolutionary explanation, and this is an evolutionary journal.

Q: Seriously, though, this wasn’t very novel or surprising (even to the researchers). Why is this research international news?

A: Because the Daily Mail evolved in an environment where sensationalising any research connected with sex or cancer was rewarded, and it can’t help itself.



  • There’s yet another ‘breakthrough’ test for future Alzheimers. The story is better than usual, pointing out “while the score could help to identify people for trials, it was too early to apply it as a genetic testing tool for use in the clinic.
  • Research from Yale finds that (in the US) a lot of the people who don’t accept that climate change is real do say they trust climate scientists, they just  don’t believe that most scientists believe in it. Which is actually weirder.
  • The Boston school system has switched from the Mercator projection to the Peters projection for maps in schools. Which is a step forward. But some of the coverage is repeating the myth that the Mercator projection became popular because it makes Europe look big. Here’s a guest post at on the topic by Ernie Davis, and here’s the XKCD take