Posts written by Thomas Lumley (1992)


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

May 22, 2017

How rich do you feel

From Scott Macleod, in a Stat of the Week nomination

The NZ Herald claims that a person earning the median NZ salary of USD $33,500 (equivalent) is the 55 millionth richest person in the world by income.

However, this must be wrong.

There are 300 million people in the USA alone, and their median income is higher than ours. This means that the average New Zealander wouldn’t even be the 55 millionth richest person in the USA, let alone the world.

Basically, yes, but it’s not quite as simple as that.  That median NZ salary looks like what you get if you multiply the NZ median “weekly personal income from salary and wages among those receiving salary and wages” (eg here) by 52, which would be appropriate for people receiving salary or wage income 52 weeks per year. The median personal income for NZ will be quite a lot lower, and the median personal income for the US is also lower: about USD30,240.

Even so, there are about 250 million adults (by the definition used) in the US, and nearly half of them have higher personal income than USD33500, so that still comes to over 100 million people. And that’s without counting Germany or the UK — or cities such as  Beijing and Shanghai that have more people with incomes that high than New Zealand does.  And that’s also assuming the web page doesn’t do currency conversions — which it looks from the code as if it’s trying to.

The CARE calculator must indeed be wrong, or using an unusual definition of income, or something. Unfortunately, the code for how it does the calculation is hidden; they say “After calculating the distribution of income, we then use a statistical model to estimate your rank.” 

As a cross-check, Pew Global also has a web page based on World Bank data.  It doesn’t let you put in your own cutpoints, but it says 7% of the world’s population had more than $50/day to live on in 2011.  The CARE web page thinks it’s more like 4.7% now.  The agreement does seem to be better at lower incomes, too — the estimates will be more accurate for people who aren’t going to use the calculator than for people who are.



May 20, 2017

Bright sunshiny day

Q: Isn’t Study suggests we need this first-thing in the morning a perfect example of click-bait?

A: Impressive. And what is this?

Q: This is daylight.

A: Makes sense.  And fits with the picture of someone stretching after getting out of bed.

Q: Does it fit the research?

A: Um.  Not so much. (link)

Q: Not people?

A: No, it was people. It’s just it was light exposure in office buildings.

Q: And these buildings weren’t where people slept?

A: No, that would be potentially inappropriate. It was where they worked.

Q: But giving people more light helped with sleep and depression?

A: “the study did not include a lighting intervention”

Q: So they compared people who had offices with windows and natural light to everyone else?

A: Basically.

Q: And there was a difference in how much sleep they got?

A: No.

Q: In whether they woke up a lot?

A: Not really. The ‘sleep efficiency’ was pretty much the same.

Q: In what, then?

A: In how long they took to fall asleep.

Q: And the depression and stress?

A: Well, the differences were statistically detectable, but they weren’t all that big.

Q: But wouldn’t you expect people with windows in their offices to be happier?

A: Yes. It’s a bit surprising how small the differences were in this study.

Q: So the headline is a bit exaggerated?

A: It’s worse than that. The headline says the research is about what you should have been doing, but it’s actually about what your employer should be doing.

May 17, 2017


  • From the NY Times: In a survey of geographical knowledge and attitudes to North Korea, Americans who can tell their arse from their elbow are more likely to favour diplomacy.  This is different from the Agrabah question, because survey participants aren’t being lied to.
  • Perceptions of bias — more precisely, claims about perceptions of bias — are very different between Democrats and Republicans in the US, according to analysis at 538.  Democrats are likely to say they think whites and Christians don’t get discriminated against much but blacks, Muslims, immigrants, gays & lesbians do. For all the groups, about a third of Republicans say they think there’s a lot of discrimination.
  • Difficulties with doing randomised experiments on social issues, from the Brookings Institution.  One of the big problems is that there isn’t good theory to allow the results of an experiment to be generalised, in contrast to drug trials where we have a pretty reasonable idea of what it means when a drug does well in a randomised trial population.  A lot of the difficulties do generalise to public health interventions, though. On a related note, economist Noah Smith talks about the role of theory and experiment in economics and in science.
  • I wrote last year about judges interrupting each other in the US Supreme Court and whether it depended on gender — the analysis in the media had ignored how much each judge talked.  There’s now an analysis with more variables (and now the right link), and the gender difference looks stronger.
May 14, 2017

There’s nothing like a good joke

You’ve probably seen the 2016 US election results plotted by county, as in this via Brilliant Maps

It’s not ideal, because large, relatively empty counties take up a lot of space but represent relatively few people.  It’s still informative: you can see, for example, that urban voters tended to support Clinton even in Texas.  There are also interesting blue patches in rural areas that you might need an atlas to understand.

For most purposes, it’s better to try to show the votes, such as this from the New York Times, where the circle area is proportional to the lead in votes

You might want something that shows the Electoral College votes, since those are what actually determines the results, like this by Tom Pearson for the Financial Times

Or, you might like pie charts, such as this one from Lisa Charlotte Rost


These all try to improve on the simple county map by showing votes — people — rather than land. The NYT one is more complex than the straightforward map; the other two are simpler but still informative.


Or, you could simplify the county map in another way. You could remove all the spatial information from within states — collecting the ‘blue’ land into one wedge and the ‘red’ land into another — and not add anything. You might do this as a joke, to comment on the President’s use of the simple county map

The problem with the Internet, though, is that people might take it seriously.  It’s not completely clear whether Chris Cillizza was just trolling, but a lot of people sure seem to take his reposting of it seriously.

May 10, 2017

Bombs away

Q: Did you see the Jagerbombs ‘as bad as taking cocaine’ headline in the Herald?

A: Doesn’t sound all that plausible. What’s a Jagerbomb? Is it like a Molotov cocktail?

Q: Apparently it’s Red Bull with Jägermeister.

A: Well, I don’t think I’m in any danger of drinking that.  How did you get the little dotty things over the ‘a’, by the way?

Q: If you hold down the ‘a’ key they pop up as an option (on a Mac). Like the Māori long vowels. But we nearly digress. Is a Jagerbomb really that dangerous?

A: Well, the story goes on to quote the researcher saying “I wondered if they were having a similar impact but to a lesser degree”

Q: And are they?

A: For suitable definitions of ‘same’ and ‘lesser’. And ‘were’.

Q: ಠ_ಠ

A: The research (no link given) looked at whether combining alcohol and energy drinks led to people getting injured more.

Q: That’s not the first thing you think of as a risk of taking cocaine. And how did they do the comparison? Recruit undergraduates with poor taste and give them Jagerbombs?

A: No

Q: You’re not going to say ‘mice’, are you?

A: No, what they did was go to the library and find all the previous studies of alcohol and energy drinks and injury, to review and summarise them.

Q: Is that useful?

A: It can be very useful. You can’t keep all that information in your head even if you’re a qualified Jagerbombologist.  Systematic reviews are a great innovation in modern medical research.

Q: So, how does the risk of injury compare to the risk with cocaine?

A: They didn’t look at that.

Q: Oh. Ok. So how big is the risk?

A: They didn’t come up with anything quantitative, because the previous research studies hadn’t been done in a similar enough way

Q: Did they come up with anything?

A: Yes, the results “suggest support for a relationship between increased risk of injury and [alcohol with energy drink] use”

Q: That’s not all that strong.

A: No.

Q: Was there a stronger relationship than just with alcohol on its own?

A: They say “some studies did not differentiate between injuries occurring in alcohol-only sessions relative to AmED sessions, making a comparison of the risk of injury between alcohol and AmED use impossible.

Q: Even if there is a difference, couldn’t it be that the sort of people who drink Jagerbombs or rum and Coke are different from people who drink beer, or cosmopolitans, or Sauv Blanc?

A: “Although the results remain mixed in terms of whether impulsivity or risk taking may moderate the relationship between AmED use and injury risk, there is enough evidence to warrant further exploration.”

Q: That all seems very reasonable.

A: The actual story isn’t too bad, either. Just the web-front-page headline.

Q: Wait, doesn’t the headline punctuation imply ‘as bad as taking cocaine’ is a quote? When it totally isn’t?

A: Yes. Yes, it does.


  • “What CPE—and the field—needs now are analysts. Lots and lots of analysts. And we, at least, are hiring DataNerds who want to be JusticeNerds™. With departments now coming in by the state-load, we are inundated with confidential data that needs to be interrogated so that we can answer some of the most fundamental questions in policingfrom Phil Goff (no, not that one) at the Center for Policing Equity, via
  • If someone claims your female developers are promoted less because they’re treated worse at code review, and you say “no, they’re treated worse because they’re more junior”, you’ve made the basic causal-inference error of conditioning on an intermediate consequence of your input variable.  Felix Salmon on Facebook’s example
  • “For too long social welfare has muddled along with bipartisan policies like shouting at the jobless or not helping people with mental health issues, without really checking if those methods work.” This, from, Lyndon Hood, isn’t going where you might expect.  It’s unfair, but not completely unfair.
  • Another reason ‘breakthrough’ science stories may be misleading: there was a research paper claiming fish preferentially eat microplastic pollution and are serious harmed by it. It has been retracted. There are allegations of deliberate fraud; the data were certainly not made available as the journal’s policy demanded.  If you remember a story on this, go back and see if the same media outlet covers the retraction.
  • I’ve written a few times about the bogus claim that the typical Kiwi pays no “net tax”.  In the other direction, there were stories about  “Tax Freedom Day” this week, on the basis of 34.8% of income going in tax. Yeah, nah.
  • Derek Lowe writes about a new analysis looking at solanezumab, Eli Lilly’s failed treatment for Alzheimer’s. The analysis claims that if the drug had been approved based on the early, weak signals of benefit, the cost to the US government would have been about ten billion dollars over the past four years. That would pay for a lot of trials, or for a lot of other improvements to dementia care.
  • There’s publication bias in research on stock-market patterns. Because of course there is.

This was almost too good to check

And this was too good to ask if it’s a joke:


(it is)

May 4, 2017

Summarising a trend

Keith Ng drew my attention on Twitter to an ad from Labour saying “Under National, the number of young people not earning or learning has increased by 41%”.

When you see this sort of claim, you should usually expect two things: first, that the claim will be true in the sense that there will be two numbers that differ by 41%; second, that it will not be the most informative summary of the data in question.

If you look on Infoshare, in the Household Labour Force Survey, you can find data on NEET (not in education, employment, or training).  The number was 64100 in the fourth quarter of 2008, when Labour lost the election.  It’s now (Q1, 2017) 90800, which is, indeed, 41% higher.  Let’s represent the ad by a graph:



We can fill in the data points in between:
Now, the straight line doesn’t look as convincing.

Also, why are we looking at the number, when population has changed over this time period. We really should care about the rate (percentage)
Measuring in terms of rates the increase is smaller — 27%.  More importantly, though, the rate was even higher at the end of the first quarter of National’s administration than it is now.

The next thing to notice is the spikes every four quarters or so: NEET is higher in the summer and lower in the winter because of the school  year.  You might wonder if StatsNZ had produced a seasonally adjusted version, and whether it was also conveniently on Infoshare…
The increase is now 17%

But for long-term comparisons of policy, you’d probably want a smoothed version that incorporates more than one quarter of data. It turns out that StatsNZ have done this, too, and it’s on Infoshare.
The increase is, again 17%. Taking out the seasonal variation, short-term variation, and sampling noise makes the underlying pattern clearer.  NEET increased dramatically in 2009, decreased, and has recently spiked. The early spike may well have been the recession, which can’t reasonably be blamed on any NZ party.  The recent increase is worrying, but thinking of it as trend over 9 years isn’t all that helpful.

May 3, 2017

A century of immigration

Given the discussions of immigration in the past weeks, I decided to look for some historical data.  Stats NZ has a report “A Century of Censuses”, with a page on ‘proportion of population born overseas.” Here’s the graph


The proportion of immigrants has never been very low, but it fell from about 1 in 2 in the late 19th century to about 1 in 6 in the middle of the 2oth century, and has risen to about 1 in 4 now. The increase has been going on for the entire lifetime of any NZ member of Parliament; the oldest was born roughly at Peak Kiwi in the mid-1940s.

Seeing that immigrants have been a large minority of New Zealand for over a century doesn’t necessarily imply anything about modern immigration policy — Hume’s Guillotine, “no ought deducible from is,” cuts that off.  But I still think some people would find it surprising.


April 28, 2017

Trends and pauses

There’s a story at the Guardian about whether there has been a ‘pause’ and an ‘acceleration’ in global warming.  The underlying research paper actually puts the question more clearly

While it is clear and undisputed that the global temperature data show short periods of greater and smaller warming trends or even short periods of cooling, the key question is: is this just due to the ever-present noise, i.e. short-term variability in temperature? Or does it signify a change in behavior, e.g. in the underlying warming trend?

Models for climate change predict that annual mean surface temperature should be going up fairly smoothly, so that the trend over a decade or so looks like a straight line. A deviation from this trend might indicate important factors have been left out of the model, or might indicate that the background processes are changing (eg as ice sheets retreat).  If you look at the recent past, compared to a straight-line trend, the observed data dipped below the straight line for a few years and have now caught up.  This raises the question of whether either of these indicated an important change in the underlying processes or a new inadequacy of the models.

To start with, let’s establish that we’re not talking about measurement error here.  The variability of annual mean temperatures around the straight-line trend isn’t like the variability of opinion poll results around a trend. The observed data are the truth.  The world really did warm less for a couple of years; it really has warmed more since then.  The straight line trend omits many factors that we know are relevant: events such as volcanic eruptions that affect the incoming sunlight, and events such as El Niño that affect the balance between air and ocean warming.

The question is whether the straight line trend is changing (fast enough to worry about).  You might reasonably object that the annual mean temperatures are far too crude to make that sort of decision; that you need much more details and more sophisticated modelling. As it turns out, you’d be right. However, the crude appearance of a slowdown and speedup in the annual means has been the fuel for a lot of discussion, so it’s worth evaluating.

What the research paper did was to model the deviations from the straight line trend as a simple random process, ignoring any year-to-year correlation.  The researchers could then evaluate mathematically how likely we would be to see an apparent pause or acceleration in warming with that amount of random variation, if in fact the trend was a perfect straight line. The deviations we have seen in the recent past are no larger than you’d expect just from the variation around a constant trend.

To be clear, this doesn’t mean there have been no changes in the trend.  In fact, we know that El Niño does cause systematic changes.  What it means is that the annual mean temperatures alone aren’t enough information to tell us about changes over a period as short as a few years. You shouldn’t change your beliefs (in any direction) over data like that. If the ‘hiatus’ had gone on for a decade, it would have meant something. If the acceleration goes on for a decade, it will mean something. But two or three years isn’t long enough to say anything.  It’s like looking at a month of data on road deaths: you can’t — or at least shouldn’t  — say much.

April 27, 2017

On debates about data

On Wednesday, the NZ Herald website featured a story and graphics by Harkanwal Singh and Lincoln Tan on immigration. This story was based on permanent and long-term migration data from Statistics New Zealand. The graphics allowed readers to explore the data for themselves. The data source was accurately described and was well targeted to the current political discussion about changing immigration policies.

The specific data set and visualisation used are not the only possible ones, and reasoned criticism of the data and analyses is entirely legitimate. StatsChat encourages that sort of thing. We have done it ourselves, and we have published links when other people do it.

Winston Peters, however, claimed that the Herald story was “fake news” and attributed the conclusions to the reporters being Asian immigrants themselves. The first claim is factually incorrect; the second (in the absence of convincing evidence) is outrageous.

James Curran (Professor of Statistics)
Thomas Lumley (Professor of Statistics)
Chris Triggs (Professor of Statistics)