Posts filed under Evidence (62)

March 26, 2014

Are web-based student drinking interventions worthwhile?

Heavy drinking and the societal harm it causes is a big issue and attracts a lot of media and scholarly attention (and Statschat’s, too). So we were interested to see today’s new release from the Journal of the American Medical Association. It describes a double-blind, parallel-group, individually-randomised trial that studied moderate to heavy student drinkers from seven of our eight universities to see if a web-based alcohol screening and intervention programme reduced their unhealthy drinking behaviour.

And the short answer? Not really. But if they identified as Māori, the answer was … yes, with a caveat. More on that in a moment.

Statistician Nicholas Horton and colleagues used an online questionnaire to identify students at Otago, Auckland, Canterbury, Victoria, Lincoln, Massey, and Waikato who had unhealthy drinking habits. Half the students were assigned at random to receive personalised feedback and the other students had no input. Five months later, researchers followed up with the students on certain aspects of their drinking.

The overall result? “The intervention group tended to have less drinking and fewer problems then the control group, but the effects were relatively modest,” says Professor Horton. The take-away message: A web-based alcohol screening and intervention program had little effect on unhealthy drinking among New Zealand uni students. Restrictions on alcohol availability and promotion are still needed if we really want to tackle alcohol abuse.

But among Māori students, who comprise 10% of our national uni population, those receiving intervention were found to drink 22% less alcohol and to experience 19% fewer alcohol-related academic problems at the five-month follow-up. The paper suggests that Māori students are possibly more heavily influenced by social-norm feedback than non-Māori students. “Māori students may have a stronger group identity, enhanced by being a small minority in the university setting.” But the paper warns that the difference could also be due to chance, “underscoring the need to undertake replication and further studies evaluating web-based alcohol screening and brief intervention in full-scale effectiveness trials.”

The paper is here. Read the JAMA editorial here.




March 18, 2014

Your gut instinct needs a balanced diet

I linked earlier to Jeff Leek’s post on, because I thought it talked sensibly about assessing health news stories, and how to find and read the actual research sources.

While on the bus, I had a Twitter conversation with Hilda Bastian, who had read the piece (not through StatsChat) and was Not Happy. On rereading, I think her points were good ones, so I’m going to try to explain what I like and don’t like about the piece. In the end, I think she and I had opposite initial reactions to the piece from on the same starting point, the importance of separating what you believe in advance from what the data tell you. (more…)

January 15, 2014

Fancy packaging of plain packaging impact

The Sydney Morning Herald has a story on the impact of plain packaging for cigarettes in Australia.  Cancer researchers in Sydney found a big spike in calls to Quitline after the packaging change, and interpreted this as evidence it was working

The researchers said although the volume of calls to Quitline was an ”indirect” measure of people’s quitting intentions and behaviour, it was more objective than community surveys where people can answer questions in a socially desirable and biased way.

On the other side, tobacco companies say there hasn’t been any actual fall in smoking.

”In November 2013, a study by London Economics found that since the introduction of plain packaging in Australia there has been no change in smoking prevalence … What matters is whether fewer people are smoking as a result of these policies – and the data is clear that overall tobacco consumption and smoking prevalence has not gone down,” he said.

In this setting you might reasonably be concerned that either side is putting their results in fancy packaging. So what should you believe?

In fact, the claims are consistent with each other and don’t say much either way about the success of the program.  If you look at the research paper, they found an increase peaking at about 300 calls per week and then falling off by about 14% per week. That works out to be a total of roughly 2000 extra calls attributed to the packaging change, ie, just over half a percent of all smokers in Australia, or perhaps a 10% increase in the annual Quitline volume. If the number of people actively trying to quit by methods other than Quitline also goes up by 10%, you still wouldn’t expect to see much impact on total tobacco sales after one year.

The main selling point for the plain packaging (eg) was that it would prevent young people from starting to smoke. That’s what really needs to be evaluated, and it’s probably too early to tell.


[Update: Of course, other countries that were independently considering changing their policies shouldn't wait for years just because Australia started first. That would be silly.]

[Update: the Quitline data are just for NSW; so perhaps 1.5% of smokers]

December 23, 2013

Meet Callum Gray, Statistics Summer Scholar 2013-2014

Every year, the Department of Statistics at the University of Auckland offers summer scholarships to a number of students so they can work with our staff on real-world projects. We’ll be profiling the 2013-2014 summer scholars on Stats Chat. Callum is working with Dr Ian Tuck on a project titled Probability of encountering a bus.  

Callum (right) explains:

“If you encounter a bus on a journey, you are likely to be exposed to higher levels of pollution. I am trying to find the probability of encountering a bus and how many you will encounter when you travel from place A to place B, taking into account variables such as the time of day and mode of transport.


“This research is useful because it will give us more of an understanding about the impact that buses have on our daily exposure to pollution. we can use this information to plan journeys and learn more about an issue that is becoming more and more apparent.

“I was born in Auckland and have lived here my whole life. I just finished my third year of a Bachelor of Commerce/Bachelor of Science conjoint majoring in Accounting, Finance, and Statistics, which I will finish at
the end of 2014.

“Statistics appeals to me because it is used everyday in conjunction with many other areas. It is very useful to know in a lot of workplaces, and it is interesting because it has a lot of real-life applications.

“I am going to Napier for Christmas and Rhythm and Vines for New Year. In the rest of my spare time, I will be playing cricket and golf, as well as hanging out with friends.”



December 2, 2013

Don’t be scared of WiFi

From TV One Breakfast this morning “WiFi detrimental to health, study suggests“. (I didn’t see this live; the Science Media Centre contacted me for comment)

The guest, Mr Kasper from Safe Wireless Technology NZ, said

Overseas research has shown that a person who uses a mobile phone for a year increases their chances of getting brain cancer by 70%, according to the SWTNZ.

The ‘overseas research’ appears to be this, a case-control study of acoustic neuroma in Scandinavia. The first thing to note is that “acoustic neuroma” isn’t the same thing as “brain cancer”. Acoustic neuroma is a rare, benign brain tumour (‘benign’ means it doesn’t spread metastatically), which is usually treatable, though often with long-term effects.  The researchers didn’t suggest that their results applied to other brain tumours; in fact, they assumed the opposite and used people with a different brain tumour, meningioma, as one set of controls for their comparisons.

The story  also says

“There’s so much research and there’s so much scientific evidence now that does more than just suggest that there is a real problem, and people are getting these problems,” Mr Kasper said.

The National Cancer Institute has a good summary of the scientific evidence, and they are not at all convinced. It certainly isn’t the case that there’s a strong association with brain cancer overall.

The new research appears to be better conducted than a lot of the past claims of associations between radio waves and health. It’s working against a strong burden of proof both from animal studies and from the fact that radio waves can’t damage DNA. I don’t think it manages that level of proof, but I think reasonable people could disagree. However, even if we assume that the association specifically with acoustic neuroma is real and causal, it doesn’t really support any concern over WiFi. Cellphones are pressed up against the ear, and so provide higher dose of radio-frequency energy to that ear. WiFi transmitters are typically not pressed up against the ear, and each doubling of the distance reduces the energy by a factor of four. And, since they don’t have to reach as far, WiFi signals are less powerful to begin with.

The story ends with

“We do want the Government to put some money into some independent research.”

I’m generally in favour of the Government putting money into research, but on this particular topic  there’s no real advantage to the research being done in New Zealand, and we have too small a population to contribute much. There are large international studies ongoing; we don’t need a small local one.

If you are worried about cellphones and acoustic neuroma, use headphones with your cellphone. If you are worried about WiFi and brain cancer, then relax.


November 27, 2013

Interpretive tips for understanding science

From David Spiegelhalter, William Sutherland, and Mark Burgman, twenty (mostly statistical) tips for interpreting scientific findings

To this end, we suggest 20 concepts that should be part of the education of civil servants, politicians, policy advisers and journalists — and anyone else who may have to interact with science or scientists. Politicians with a healthy scepticism of scientific advocates might simply prefer to arm themselves with this critical set of knowledge.

A few of the tips, without their detailed explication:

  • Differences and chance cause variation
  • No measurement is exact
  • Bigger is usually better for sample size
  • Controls are important
  • Beware the base-rate fallacy
  • Feelings influence risk perception
November 24, 2013

Measuring the right variable



And, of course, New Zealand won the Rugby world cup, NZ or Australia will win the League world cup next weekend, and NZ was the only undefeated team in the last soccer World Cup, so Europe’s overall football credentials seem to be on shaky ground.  At least if you set up the comparison very, very carefully.

Generalisations of this approach to unsupported nutritional advertisting, surrogate outcomes in clinical trials, ranking of universities, and the claim that people with incomes below $110,000 collective pay no net income tax provide a lot of the work  for StatsChat.

(via Ben Atkinson on Twitter. I can’t identify the original source, but this version has the best punctuation.)



November 19, 2013


  • Animated visualisation of motor vehicle accident rates over the year in Australia. Unfortunately it’s based on just one year of data, which isn’t really enough. And if you’re going the effort of the animation, it would have been nice to use it to illustrate uncertainty/variability in the data
  • Randomised trials outside medicine: the combined results of ten trials of restorative justice conferences. Reoffending over the next two years was reduced, and the victims were happier with the handling of the case. (via @hildabast)
  • How much do @nytimes tweets affect pageviews for their stories?
November 5, 2013

Can we bring out the real numbers now?

So, the decision has been made and the blood alcohol limit will be lowered.  Perhaps now we can start using realistic numbers for the impact.  The story in the Herald today shows the problem, although it’s actually much better than anything I’ve seen in the mainstream media previously:

The changes come after a two-year review of the impact of lowering the legal blood alcohol limit by 30mg suggested 3.4 lives would be saved a year and 64 injury-causing crashes avoided.

It would also save $200 million in social costs over 10 years.

“Alcohol impairment is a major cause of road accidents in New Zealand, with an average of 61 fatalities, 244 serious injuries, and 761 minor injuries every year caused by at-fault drivers who have been drinking,” said Transport Minister Gerry Brownlee.

“The social cost of these injuries and fatalities is $446 million – a huge sum in a country of our size.”

In the first paragraph the estimated benefit based on actual research is quoted. That’s a big step forward. The second paragraph is just wrong: the social costs aren’t in addition to the lives saved and injuries prevented; that’s where the social cost numbers come from. And it’s multiplied by ten years.

In the third and fourth paragraphs Mr Brownlee is quoted as justifying the change by quoting total costs of drink driving. The social cost number in the fourth paragraph is 22 times larger than the actual estimated benefit. You’d think that sort of discrepancy would draw some journalistic comment.

And later in the story we are told about a victim of a drunk driver. A driver whose blood alcohol concentration was 190mg/100ml, more than twice the existing legal limit, and who was duly convicted and sent to prison under the old laws. Not the sort of person whose behaviour is likely to be affected by this change.

October 24, 2013

Burning issue

I’m in Sydney at the moment, so this is topical, as well as being an illustration of maps, infographics, and internet fact-checking.

From Paul Rosenzweig on Twitter, allegedly a map of the bushfires shown on NBC News in the US

attributed to NBC News

People in Australia think this map is hilarious/outrageous depending on personality — the current emergency was just in New South Wales.  That was my reaction too. But the NBC News blog gets this right, which is a bit confusing

However, @Aus_ScienceWeek, the people who run National Science Week, point out that the map looks rather like the appropriate subsection of NASA’s satellite-based fire map from mid-September



so it might well be correct in the sense that there actually fires in those places, though still wrong as a description of the emergency.