Posts written by Thomas Lumley (1805)


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

June 19, 2016

Selfies can damage your credibility

A Herald story, under the headline Selfies can damage your health

Dermatologists now believe that regularly exposing the face to the light and electromagnetic radiation from smartphones can speed up ageing and wrinkles.

The people making that claim don’t seem to be giving any evidence. You’d need a lot of evidence to believe a claim like that, given how much more light and electromagnetic radiation we get from the sun. Sure, sunlight doesn’t have quite the same frequency spectrum, but it’s so much brighter that it will be giving you more exposure at any frequency.

Also, when you’re taking selfies, your phone is usually not particularly close to your face.  After all, that’s why selfie sticks were invented. If the claim were true, the real message would be that selfies protect your health.

Doctors even claim they can tell which hand a person holds their phone in just by looking at which side of the face is most damaged.

I have no real doubt that this doctor can tell which hands his patients hold their phones in — for example, by observing what hand they use for other things, or what pocket the phone is kept in.  But, again, people nowadays don’t spend much time holding their phones up to one side of their faces. You only do that for phone calls, and only then if you don’t have a headset, and that’s such a twentieth century way to use a phone.

And, yes, the people involved are all selling something.

(This story comes from the Daily Telegraph, though it didn’t say so when I first read it, and according to a credible witness it said Daily Mail earlier)

June 18, 2016

Why headlines matter

From the Herald

According to a new study by computer scientists at Columbia University and the French National Institute, 59 per cent of links shared on social media have never actually been clicked: In other words, most people appear to retweet news without ever reading it.

Perhaps appropriately, the Herald passed on this story from the Washington Post without passing on their link to the research.

If you were reading carefully, you might notice the last part of the paragraph doesn’t actually fit the rest: it isn’t that ‘most people‘ retweet news without ever reading it, it’s that most retweets are done without reading (or before reading).  Even with that caveat, the research says that headlines matter — and I’m going to keep complaining about them.

The main focus of the research was something different, though. They were comparing clicks on the ‘primary’ URLs from the media site itself with the clicks on shortened URLs produced by readers.  Although the primary URLs had the majority of ‘impressions’, the secondary URLs added up to more clicks.  That is, when news stories are actually read via Twitter, more often than not it’s because of a personal recommendation by someone else who has actually read the story.

June 16, 2016


A nice cuppa

So, there’s a new IARC cancer monograph out, but this one is disappointingly un-scary. (Here’s the Q&A)

There are two basic points. First, coffee looks safe. Officially it’s “inadequate evidence”, but that basically means “if we were in the business of calling things ‘safe’, coffee would be on the list”.  There has been enormous effort over decades to find health risks of coffee for all sorts of diseases, with extremely limited success. Coffee can make it harder to fall asleep, and that’s about it.

The second point is about ‘very hot drinks’. These are listed as ‘probably carcinogenic’, meaning there’s some evidence that regular consumption increases the risk of oesophageal cancer  (not that you’ll probably get cancer if you drink them).  By ‘very hot’, they mean very hot,  over 65°C when consumed, which is relatively unusual in New Zealand.  Here, it’s alcohol and smoking that are the main risks for oesophageal cancer.


June 15, 2016

Counting refugees

The Immigration Minister, Michael Woodhouse, was on RadioLive

Per capita, New Zealand is ranked about 90th in the world for the number of refugees it accepts. Mr Woodhouse says this low ranking isn’t the Government’s fault — its quota puts New Zealand at “about seventh or eighth”. It’s refugees turning up unannounced that puts other countries ahead — like Jordan taking in refugees from the Syrian civil war, for example.

“I don’t want to get into a game of statistics, but we can be pretty pleased with what. We can say seventh, you can say 90th.”

When you have answers that differ that much, it’s either because someone has the numbers wrong, or (as in this case) because they are answers to different questions. When you have answers to different questions, the right statistical approach isn’t to look at the answers and decide which one you like, it’s to look at the questions and ask which is the right one.  That is, should we regard refugee numbers as basically about resettlement quotas managed through UNHCR, or as including asylum seekers?

People setting off on their own into neighbouring countries is, has always been, and probably always will be the main way they become refugees. The refugee resettlement process is a way for countries who aren’t within walking/driving/sailing distance of a humanitarian disaster to contribute and to perform some of their duty to the international community.  But the refugees who turn up unannounced, not the tidily resettled ones, are the primary and usual case. They should be included in the count unless there’s some special reason occasionally not to.

New Zealand doesn’t accept many refugees — and most countries probably wouldn’t if they could avoid it. Germany and Turkey are currently handling very high numbers because of asylum seeker fleeing the Syrian crisis, but had much lower numbers ten years ago. But German and Turkey really are accepting more refugees now. If you were in Mr Woodhouse’s ‘game of statistics’ and you were handed the ‘no, nothing has changed much’ side of that argument, you would lose.

New Zealand accepts a very low number of refugees either per capita or per unit GDP.  Some people are happy with that. Many people, probably, are happy that it’s not ten or twenty times larger, as in several European countries. But it is a very low number and people who don’t want it to change should be honest about that.

In any case, whether we count qualifying asylum-seekers or just resettled refugees, whether we look per capita or per dollar of GDP, there’s one consistent finding. We accept fewer refugees than Australia.

June 14, 2016

Why everyone trusts us

  • In the UK, there’s been a big increase in the use of National Health Service data to track illegal immigrants — this was previously just done for serious criminals. (Buzzfeed)
  • CHICAGO — In this city’s urgent push to rein in gun and gang violence, the Police Department is keeping a list. Derived from a computer algorithm that assigns scores based on arrests, shootings, affiliations with gang members and other variables, the list aims to predict who is most likely to be shot soon or to shoot someone. New York Times
  • There’s a new UK website that does detailed analysis of your social media to tell your landlord whether you’ll be able to pay your rent. “If you’re living a normal life,” Thornhill reassures me, “then, frankly, you have nothing to worry about.” (Washington Post)
  • “We don’t turn people away,” Might said, but the cable company’s technicians aren’t going to “spend 15 minutes setting up an iPhone app” for a customer who has a low FICO score.  (fiercecable, via
  • Another startupclaims it can “reveal” your personality “with a high level of accuracy” just by analyzing your face, be that facial image captured via photo, live-streamed video, or stored in a database. It then sorts people into categories; with some labels as potentially dangerous such as terrorist or pedophile,” (also via
June 13, 2016

Reasonable grounds

Mark Hanna submitted an OIA request about strip searches in NZ prisons, which carried out with ‘reasonable grounds to believe’ the prisoner has an unauthorised item.  You can see the full response at FYI. He commented that 99.3% of these searches find nothing.

Here’s the monthly data over time:

The positive predictive value of having ‘reasonable grounds’  is increasing, and is up to about 1.5% now. That’s still pretty low. How ‘reasonable’ it is depends on what proportion of the time people who aren’t searched have unauthorised items: if that were, say, 1 in 1000, having ‘reasonable grounds’ would be increasing it 5-15-fold, which might conceivably count as reasonable.

We can look at the number of searches conducted, to see if that tells us anything about trends
Again, there’s a little good news: the number of strip searches has fallen over the the past couple of years. That’s a real rise and fall — the prison population has been much more stable. The trend looks very much like the first trend upside down.

Here’s the trend for number (not proportion) of searches finding something
It’s pretty much constant over time.

Statistical models confirm what the pictures suggest: the number of successful searches is essentially uncorrelated with the total number of searches. This is also basically good news (for the future, if not the past): it suggests that a further reduction in strip searches may well be possible at no extra risk.

June 12, 2016

Walking it back

First the headline (Herald, reprinting Daily Telegraph)

Finger-prick test that can show risk of diabetes

then it’s “on the horizon”

The finger-prick test, which could be available at GP surgeries or even chemists, looks for molecules in the blood that indicate diabetes is developing.

but in the present tense. Then

The specific biomarkers involved are being kept a closely guarded secret for now, but once a prototype test has been developed, trials will take place.

This ‘test’ not only hasn’t been evaluated in real patients; it doesn’t even exist yet.


Currently, doctors can test for diabetes only by taking blood glucose readings that show whether the disease is already present.

That’s only true if you squint from exactly the right angle. Since 2012, testing for HbA1c, a byproduct of elevated glucose, has been a general screening recommendation in NZ and one of the public health performance indicators. One of the reasons given for screening is

effective screening aims to reduce the incidence of diabetes through detection of people with pre-diabetes

You could argue ‘the disease is already present’ in people with pre-diabetes, but not in the sense that’s relevant to screening.

The current test isn’t all that good, and perhaps when they finish inventing it the new one will be better. But it’s not a test yet; it’s not a ‘health’ story yet; and with so little disclosed information it’s not clear that it’s even a science story yet.

June 11, 2016


  • from Wired: Transitland collects and aggregates transit data feeds from around the world. Though apparently not this part of the world
  • How do you study the impact of changes from the ‘sharing economy’?  Andrew Gelman and Tom Slee on AirBNB “Academics now tend to ask me (a) how to refer to the data set, and (b) how they can validate it for publication purposes, to which I generally reply (a) I have no idea, and (b) I have no idea”. Sarah Emerson on Uber

Crimes and data limitations

There’s another neat interactive map at Herald Insights, this time of meshblock-level data on assaults in public places reported to the police. It’s a good job, but I still want to talk about some of the limitations forced on the graphics by the available data.

The most obviously strange thing about the map is the hospitals: in Auckland, two of the big hotspots are Auckland City Hospital and Middlemore Hospital. When we talk about hospitals being dangerous places, we usually mean surgical errors or drug-resistant bacteria, not assault. There are two relatively boring reasons that the hospitals stand out, which they share with the town and city centres: the hospitals have a lot of people in them, and they are public places.

In the Herald map, the colour coding reflects the number of reported victimisations, not a rate per capita.  That’s because the right population of capitas is hard to define and not readily available.  We know the number of people who live in each meshblock, and the number who work in each meshblock, but the assaults are those in public places. When you’re at home — and for many people, when you’re working — you aren’t in a public place.   There isn’t easily-available data on the number of people in public places, at meshblock resolution, for the whole country.

The other question the hospital data should raise when talking about the most dangerous areas is ‘dangerous to whom’? Who gets assaulted in hospitals?  I don’t know, but I’d expect staff are at reasonably high risk, and so are people who have already been fighting before they arrived.  It might well be that hospitals are pretty safe for people just going there because their chest hurts or they decided to mow the lawn barefoot.