Posts written by Thomas Lumley (1824)


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

July 22, 2016

Abstract isn’t the same as logical

Suppose, to copy a classic example, you are a checking license compliance for a pub and have to  make sure only people 18 or older are drinking alcohol.  There are four people present. Alice is drinking beer. Boris is drinking water. Chris is fifteen. Doris is 50. Do you need to:

For most people, this is pretty easy.

The Herald has an equivalent puzzle that has been made pointless and abstract, in terms of letters and numbers, and lots of people get it wrong. That’s fine, except the headline is “Card test reveals how logical you are.”  Manipulating conditional implications abstractly is a useful specialised skill, but it’s not the same as logic.  In a similar way, manipulating probabilities symbolically is a useful specialised skill, but it’s not the same as understanding risk.

When it’s just a game, as in the story, this isn’t a big deal. But when you have a real question, communicating it so that it’s easy to answer rather than pointlessly hard does matter.


July 21, 2016

The ‘breakthrough’ story

This is the front page of The Age, Melbourne’s serious newspaper, today:


As Jack Scanlan observed on Twitter, it’s great to see science on the front page. On the other hand, the story is an example of how the ‘breakthrough’ narrative dominates science stories (as Scanlan himself wrote in Lateral magazine back in February).

The description of the research itself is fine, but the impact is a bit overplayed

“The idea is to screen people who aren’t displaying symptoms,” Dr Cheng said. “We can then identify their risk of developing Alzheimer’s disease and intervene earlier.”

Eventually that’s going to work, but right now we don’t know how to intervene and so there’s not much point in doing it earlier.

From the viewpoint of journalism, though, there’s another issue. Just in New Zealand papers over the past few years we have:

  • Has a 15-year-old found a way to test for Alzheimer’s? (Herald, 7/2015)
  • Blood test could detect dementia (Herald, 3/2014)
  • Blood test could give ten year warning of Alzheimer’s (Herald, 6/2015)
  • Alzheimer’s blood test hope (Herald, 7/2014)
  • Excitement over Alzheimer’s discovery (Otago Daily Times, 4/2016)

The last one even uses the same approach, measuring microRNAs in blood samples.  It’s a good idea; it’s good science; it may eventually be useful. It’s not a unique breakthrough.

July 20, 2016

Another set of rugby predictions

The Herald has a new set of rugby team ratings going back into history, with pretty graphs as well, based on work by UoA student Wil Undy.  These are ‘Elo’ ratings in the modified sense that uses the term. The original Elo method was for chess, where you only get a winner, not a margin of victory, but it’s been updated to use the extra information from the winning margin.

So, how are these different from the StatsChat ratings?   The methods are fairly similar: there’s a rating for each team, which is updated using the results of each game, and there’s a tuning parameter that controls how much each new game is allowed to change the rating.

The primary difference is how the ratings are calibrated. In David Scott’s system the difference in ratings estimates the margin; in an Elo system the difference in ratings can be converted into a predicted probability of winning.

Just based on crude probability of getting the right winner, the StatsChat predictions may be very slightly better — for the last three seasons of Super 15, David Scott has got 65%, 66%, and 68% right, and the Herald claims 64-65% for Will Undy’s model.  On the other hand, if you actually want to bet, a predicted probability could be more useful than a predicted margin.

The graph for the Crusaders shows one interesting feature of the model


The Crusaders’s rating has improved through the season in every season for the past 15 years, suggesting that the between-season correction is too strong, or that memory for more than one year into the past might be helpful.


Rugby fans will be able to find other interesting patterns and places where tweaks could be made. What’s interesting about both Wil Undy’s new method and David Scott’s approach is how well you can do with a formula that knows nothing about rugby or rugby players and only remembers one number for each team.  These models are most interesting as a baseline: how much better can you do by following the news and taking advantage of actual rugby knowledge?

July 19, 2016

Good news

Three pieces of statistical good news:

First, from the New York Times,  several serious diseases are getting less common, and we don’t really know why. Heart disease rates have been falling for my whole lifetime. More recently, rates of dementia have been going down (though the number of cases is going up, because there are more old people). Hip fractures are down. And now colon cancer is getting less common.  In all these cases there are well-understood contributing factors, but these aren’t enough to explain the decrease (except perhaps for heart disease).

Violence is also down: in the specific politically-relevant case, fewer US police officers have been shot during the Obama presidency than the equivalent periods of Clinton, Bush, or Reagan.

And finally, in US states where medical marijuana has been legalised, there are fewer prescriptions written for other competing drugs. (Washington Post, original research paper). This is most dramatic in prescriptions for painkillers: marijuana is competing successfully against opioid narcotics.

Polls over petitions

I mentioned in June that Generation Zero were trying to crowdfund an opinion poll on having a rail option in the Auckland’s new harbour crossing.

Obviously they’re doing this because they think they know what the answer will be, but it’s still a welcome step towards evidence-based lobbying.

The results are out, in a poll conducted by UMR. Well, a summary of the results is out, in a story at The Spinoffand we can hope the rest of the information turns up on Generation Zero’s website at some point. A rail crossing is popular, even when its cost is presented as part of the question:


The advantage of proper opinion polls over petitions or other sort of bogus polls is the representativeness.  If 50,000 people sign a petition, all you know is that the true number of supporters is at least 50,000 (and maybe not even that).  Sometimes there will be one or two silent supporters for each petition vote (as with Red Peak); sometimes many more; sometimes fewer.

Petitions do have the advantage that you feel as if you’re doing something when you sign, but we can cope without that: after all, we still have social media.

July 18, 2016


  • Lovely rant about Aloe vera gel and sunburn: “Listen, you antiscience monster: These people who sell aloe vera are stealing from you. As of 2004, the market for finished aloe products was worth $110 billion. The aloe barons are taking your money and building gigantic aloe palaces and not helping your sunburn at all. They are probably taking long soaks in great big hot tubs for which you helped pay. And they’re probably wearing lots of sunscreen, too, because they know nothing in their cabinets can heal sunburn.”
July 4, 2016

Run away

There’s a piece in the Herald, from The Conversation (with attribution, and links intact, yay!), on the supposed brain benefits of running.  There’s an interesting evolutionary speculation that humans, specifically, might benefit from running because of our pre-history of long-distance running to hunt

The growth of new brain cells in the hippocampus and the enhancement of spatial memory that is brought on by endurance running is basically an evolutionary safety net for when you have outrun your knowledge, when you have run so far that you no longer know where you are and you need to learn, fast. It is a mechanism that makes information uptake easiest when historically you might have been tired, lost, and at your most vulnerable.

There’s also some actual evidence on brain cell growth

What the new research tells us is that it is not just any exercise that will create new brain cells for you. In the study by Finnish researchers, they discovered that only certain kinds of exercise are likely to result in the growth of new brain cells in adults.

As you will find if you follow the link (but not from reading the story), that research is in rats. Unless their ancestors, too, pursued their prey across miles of African savannah, it rather tends to undermine the evolutionary argument.

July 3, 2016


July 2, 2016

Dark and full of terrors

Two items from the Herald this week:

70-year-old licked by dog nearly dies from blood poisoning

One paracetamol in pregnancy could raise risk of autism

There has long been an argument over whether the daily news reports should show more good news (that isn’t sports). Supporters of the status quo argue that the bad news is the important news. That might be true for wars and rumors of war, but it isn’t true in stories fishing for a scary health problem.

The dog story reported a cause of sepsis that has been diagnosed about once every two years in the UK over the past quarter-century. Since there are about 30,000 cases of sepsis per year in the UK, that’s 0.00% of cases, to two decimal places.  It wouldn’t be surprising if the health benefits of having a dog were larger than this, and it’s pretty clear the benefits in happiness are.

The paracetamol story is the other sort of unnecessary health scare: something that probably isn’t true and certainly isn’t supported by enough evidence for a public-health warning.  Since paracetamol has nothing to recommend it as a recreational drug, women who take it during pregnancy are probably doing so for a good reason, and it’s going to be hard to distinguish the effects of that reason from those of the drug.

In fact, the study  (Table 2)found that mothers who took paracetamol during pregnancy had slightly lower risk of autism-spectrum symptoms in their children than those who didn’t. However, the mothers who took paracetamol looked as if they should have already have been at lower risk than those who didn’t. When the researchers attempted to control for this, there was basically the same risk of autism-spectrum symptoms in the children of those who took paracetamol as those who didn’t.

However, the basically-no-difference could be separated out into a higher level of symptoms  in boys and a lower level in girls, by a bit more than half a point on a scale where 15 points is the threshold for likely autism.  This got reported in the story as

There was also a link with paracetamol and signs of autism – but only in boys.

And the “One paracetamol” headline? There was absolutely no analysis in the paper comparing ‘one paracetamol’ to no paracetamol.

July 1, 2016

Too good to check

Twitter is a good way to get news and rumours of news from across the world, but it also exposes you to a lot of items that are either deliberate fraud or just ‘too good to check’.  Here’s one: it claims to compare maps of the ‘Leave’ vote with BSE prevalence.


It’s clear what idea the author was going for, and it’s also clear that it has to be unfounded as well as malicious. The BSE prions weren’t preferentially consumed in farming areas — people in cities eat hamburgers, too — and nvCJD is not only very rare, but primarily affects movement rather than political beliefs.

However, it’s not inconceivable that farming areas which experienced losses from BSE and then later from foot-and-mouth would be anti-government and possibly anti-European. Some correlation, even a strong correlation, would be possible for that reason.

If you cared about the truth, there’s a simple two-word Google search you could do before passing on the maps: BSE Scotland. Yes, there was mad cow disease north of the border. You could also note the implausibility of having exactly the same map layout, and a color scheme that was just a grayscale version of the modern one.