Posts written by Thomas Lumley (1299)


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

October 24, 2014

Something in the air

There’s a story “Pollution can cause lung problems in unborn baby – research” in the Herald, which I’m not  convinced by, but the reasons are relatively subtle.

The researchers compared levels of traffic-related air pollution exposure for different pregnant women, and looked at the lung function of the children at age four and a half (press release).  The story gets the name of the main pollutant (nitrogen dioxide) wrong in two different ways, but is otherwise a good summary.  It’s all correlation, but weaker associations than this are fairly reliably estimated for short-term exposures to air pollution. Long-term exposure is different, and that’s what’s interesting.

Studies of short-term effects of air pollution compare the number of people dying or going to hospital on days when pollution is high to the number on days where pollution is low.  That is, the comparisons of pollution are for the same people and for the same air pollution monitors. There are a fairly limited selection of other factors that could explain the association — the main ones being related to weather.

Studies of longer-term effects compare people with high exposure to pollution and people with low exposure to pollution.  Actually, they don’t quite do that, because air pollution monitoring is expensive in labour and equipment. They compare people with high estimated exposure and low estimated exposure. Since we’re comparing different people, any factor that affects health and also affects where people live could cause a bias, and it’s very well established that poorer people tend to get exposed to more pollution, at least in cities. Also, since we’re comparing different air pollution monitors, there can be biases from how representative the monitors are of the local area.

These problems mean that it’s much harder to be confident about effects of longer-term air pollution exposure, even though these effects are likely to be bigger than the short-term ones. Fortunately, we don’t need to be sure of these effects in setting public policy. The main source of the pollution is traffic, and there are other independent reasons why we want to have fewer cars burning less fuel.

On the statistical generalisability of personal experience

Going by people I know in real life or on Twitter, you would think the majority of people brought up in the Mormon church become scientists. though I am informed this is not actually the case.

There’s an interview with one of them, Heather Hendrickson, in the Herald.

October 23, 2014

Official Information and Open Data

In recent years it has become much easier to just go and get routine government data. It’s now easy to put data up online, and organisations do it. We might whinge about how often the URLs and layouts change, but you can get and reuse information in ways that used to be impossible. For examples in just one field, see the blog of the NZ geodata company Koordinates.

On the other hand, non-routine requests seem to be increasingly difficult. David Fisher, of the Herald, gave a talk in Wellington last week on the Official Information Act. The talk has been published at Public Address

When I started, if I wanted to know about something, I would ring and ask. For example, if I want to know about how Kauri stumps were exported, I would ring up the equivalent of the MPI and ask how Kauri stumps get exported. I would then spend half an hour on the phone to the guy who oversaw the exporting – often the guy who was physically down at the docks – and I would be informed.

It seems a novel idea now. I can barely convey to you now what a wonderful feeling that is, to be a man with a question the public wants answering connecting with the public servant who has the information.

Things have changed, he says.

October 22, 2014

Screening the elderly

I’ve seen two proposals recently for population screening of older people. They’re probably both not good ideas, but for different reasons.

We had a Stat of the Week nomination for a proposal to screen people over 65 for depression at ordinary GP visits, to prevent suicide. The proposal was based on the fact that 70% of the suicides were in people who had visited a GP within the past month.  If the average person over 65 visits a GP less than about 8.5 times a year, this means those visiting their GP are at higher risk.  However, the risk is still very small: 225 over 5.5 years is 41/year, 70% of that is 29/year.

To identify those 29, it would be necessary to administer the screening question to a lot of people, at least hundreds of thousands. That in itself is costly; more importantly, since the questionnaire will not be perfectly accurate there will be  tens of thousands of positive results. For example, a US randomised trial of depression screening in people over 60 recruited 600 participants from 9000 people screened. In the ‘usual care’ half of the trial there were 3 completed suicides over the next two years; in those receiving more intensive and focused help with depression there were 2. The trial suggests that screening and intensive intervention does help with symptoms of major depression (probably at substantial cost), but it’s not likely to be a feasible intervention to prevent suicide.


The other proposal is from the UK, where GPs will be financially rewarded for dementia diagnoses. In contrast to depression, dementia is pretty much untreatable. There’s nothing that modifies the course of the disease, and even the symptomatic treatments are of very marginal benefit.

The rationale for the proposal is that early diagnosis gives patients and their families more time to think about options and strategies. That could be of some benefit, at least in the subset of people with dementia who are able and willing to talk about it, but similar advance planning could be done — and perhaps better — without waiting for a diagnosis.

Diagnosis isn’t like treatment. As a British GP and blogger, Martin Brunet, points out

We are used to being paid for things of course, like asthma reviews and statin prescribing, and we are well aware of the problems this causes – but at least patients can opt out if they don’t like it.

They can refuse to attend a review, decline our offer of a statin or politely take the pill packet and store it unopened in the kitchen cupboard. They cannot opt out of a diagnosis.


Infographic of the week

From the twitter of the Financial Times, “Interactive: who is the better goalscorer, Messi or Ronaldo?”

I assume on the FT site this actually is interactive, but since they have the world’s most effective paywall, I can’t really tell.

The distortion makes the bar graph harder to read, but it doesn’t matter much since the data are all there as numbers: the graph doesn’t play any important role in conveying the information. What’s strange is that the bent graph doesn’t really resemble any feature of a football pitch, which I  would have thought would be the point of distorting it.



The question of who has the highest-scoring season is fairly easy to read off, but the question of “who is the better goalscorer” is a bit more difficult. Based on the data here, you’d have to say it was too close to call, but presumably there’s other information that goes into putting Messi at the top of the ‘transfer value’ list at the site where the FT got the data.

(via @economissive)

October 20, 2014

Advertising about your weekend

Today’s Daily Mail story in the Herald is unusual, not because it’s a survey done to advertise a company, but because the company of that name in New Zealand is getting a freebie. The story is describes people lying about their boring weekends, and it’s a survey commissioned by Travelodge, the UK budget hotel chain. The hotel company with with the Travelodge brand in this part of the world is, as far as I can tell, not related.

What is notable about the story, which confused me at first when looking across multiple versions in the British media, is that it’s a re-run. Travelodge did the same survey in 2011, on a larger sample. Here’s the Mail story from last time; the Herald escaped it then.

The press release for this year’s survey isn’t up, but if it’s like the 2011 one it won’t give any information about how the survey was conducted, and only reports a few highlights of the results, so if it were about anything important you wouldn’t want to pay attention.

October 19, 2014

Broadening your data display palate — multivariate beer?

Nathan Yau at Flowing Data has a project page on multivariate beer. That is, he wants to use beer recipes to encode information about US counties taken from the American Community Survey:

The great thing about beer is that it has plenty of dimensions to work with: body, bitterness, head retention, hop profile, color, aroma, alcohol by volume, and plenty more. The amount of various ingredients affects how beer looks, tastes, and smells.

Still a work in progress, here’s how a beer recipe is formed.

  • Greater head retention should increase with higher education, so a grain called Carapils is added.More hop aroma represents higher employment. This comes from more hops at the end of a boil and dry hopping.
  • Rye adds spice and complexity to the beer as health care coverage increases.
  • A darker-colored and more full-bodied beer comes from higher median household income and Crystal Malt 40.
  • More hop bitterness and flavor means more people per square mile, and the type of hops — Cascade, Centennial, Citra, Warrior, and Magnum — represents the races of the population.

That sounds fun, but I’m not convinced by its possibilities for data communication.

People often want to use other senses than vision for data communication, because they would provide more dimensions.  There are a couple of problems with this. First, the bandwidth and resolution of the other senses aren’t as good — for example, even a professional tea-taster can’t manage much over a thousand data points per day. Second, there’s encoding: the idea is to take advantage of the richness of experience from using all the senses, but it’s hard enough to work out how to encode numbers visually, and it will be much harder to come up with encodings for the other senses that convey accurate quantitative information.

October 18, 2014


1. There’s a conference coming up in Canada on “Fairness, Accountability, and Transparency in Machine Learning”, a topic I wrote a little about for the Listener

Questions to the machine learning community include:

  • How can we achieve high classification accuracy while eliminating discriminatory biases? What are meaningful formal fairness properties?
  • How can we design expressive yet easily interpretable classifiers?
  • Can we ensure that a classifier remains accurate even if the statistical signal it relies on is exposed to public scrutiny?
  • Are there practical methods to test existing classifiers for compliance with a policy?


2. From Nate Silver at

Democrats may not be wrong. The polls could very well be biased against their candidates. The problem is that the polls are just about as likely to be biased against Republicans, in which case the GOP could win more seats than expected.

This sort of slowly varying bias is probably one of the reasons the NZ election polls weren’t very good: not only did they have more variability than you’d expect given the sample sizes, but averaging didn’t cancel out much of the error.

3. Yesterday was Spreadsheet Day. Flee in terror! (via @kara_woo)

4. An informative  visualisation of what the world eats, over time. (via Harkanwal Singh)


When barcharts shouldn’t start at zero

Barcharts should almost always start at zero. Almost always.

Randal Olson has a very popular post on predictors of divorce, based on research by two economists at Emory University. The post has a lot of barcharts like this one


The estimates in the research report are hazard ratios for dissolution of marriage. A hazard ratio of zero means a factor appears completely protective — it’s not a natural reference point. The natural reference point for hazard ratios is 1: no difference between two groups, so that would be a more natural place to put the axis than at zero.

A bar chart is also not good for showing uncertainty. The green bar has no uncertainty, because the others are defined as comparisons to it, but the other bars do. The more usual way to show estimates like these from regression models is with a forest plot:


The area of each coloured box is proportional to the number of people in that group in the sample, and the line is a 95% confidence interval.  The horizontal scale is logarithmic, so that 0.5 and 2 are the same distance from 1 — otherwise the shape of the graph would depend on which box was taken as the comparison group.

Two more minor notes: first, the hazard ratio measures the relative rate of divorces over time, not the relative probability of divorce, so a hazard ratio of 1.46 doesn’t actually mean 1.46 times more likely to get divorced. Second, the category of people with total wedding expenses over $20,000 was only 11% of the sample — the sample is differently non-representative than the samples that lead to bogus estimates of $30,000 as the average cost of a wedding.

October 16, 2014

Do you feel lucky?

I’m glad to say it’s been quite a while since we’ve had this sort of rubbish from the NZ papers, but it’s still  going across the Tasman (the  Sydney Morning Herald)

If you’re considering buying a lottery ticket, you’d better make sure it’s from either Gladesville or Cabramatta, which are now officially Sydney’s luckiest suburbs when it comes to winning big. 

NSW Lotteries has released statistics that show the luckiest suburbs across all lotto games in NSW and the ACT, as well as other tips for amateurs hoping to ring their bosses tomorrow morning to say they wouldn’t be coming in to work. 

Of course, the ‘luckiest’ suburbs are nothing of the sort: just the ones where the most money is lost on the lotteries. Cabramatta has improved a lot in recent years, but it’s still not the sort of place you’d expect to see called ‘lucky’.