Stats Chat

More than a year ago, ESR scientists produced a report on drugs and alcohol found in blood samples taken after fatal crashes.  Now, the Drug Foundation is launching a publicity campaign using the data.  Their website says “Nearly half of drivers killed on New Zealand roads are impaired by alcohol, other drugs, or both.” But that’s not what the ESR report found. [Edited to add: the Drug Foundation is launching a campaign, but the TV campaign isn’t from them, it’s from NZTA]

The ESR report defined someone as impaired by alcohol if they had blood alcohol greater than 0.03%, and said they tested positive for other drugs if the other drugs were detectable.   If you look at the report in more detail, although 351/1046 drivers had detectable alcohol in their blood, only 191/1046 had more than 0.08%.  At 0.03% blood alcohol concentration there may well be some impairment of driving, and near 0.08% there’s quite a lot, but we can’t attribute all those crashes to alcohol impairment rather than inexperience, fatigue, bad luck, or stupidity.  At least the blood alcohol concentrations are directly relevant to impairment.  An assay for other drugs can be positive long after the actual effect wears off. For example, a single use of cannabis will show up in a blood test for 2-3 days, and regular use for up to a week.  In  fact, the summary of the ESR report specifically warns “Furthermore, it is important to acknowledge that the presence of drugs and alcohol in the study samples does not necessarily infer significant impairment.”   Regular pot smokers who are scrupulously careful not to drive while high would still show up as affected by drugs in the ESR report.  In fact, the Drug Foundation makes this distinction when they talk about random roadside drug testing, pointing out the advantages of a test of actual impairment over a test of any presence of a drug.

The Drug Foundation also did a survey of community attitudes to driving while on drugs (also more than a year ago), and it is interesting how many people think that stimulants and cannabis don’t impair their driving.  However, if you look at the survey, it turns out that it was an online poll, and “Respondents were recruited to the online survey via an advertising and awareness campaign that aimed to stimulate interest and participation in the study.” Not surprisingly, younger people were over-represented “The mean age of respondents was 38.1 years”, as were people from Auckland and Wellington. Maori, Pasifika, and Asians were all under-represented.  36% of respondents had used cannabis in the past year, more than twice the proportion in the Kiwi population as a whole.  No attempt was made to standardise to the whole NZ population, which is the fundamental step in serious attempts at accurate online polling.  [If we could use the data as a teaching example, I’d be happy to do this for them and report whether it makes any difference to the conclusions]

And while it’s just irritating that news websites don’t link to primary sources, it is much less excusable that the Drug Foundation page referencing the two studies doesn’t provide links so you can easily read them. The study reports are much more carefully written and open about the limitations of the research than any of the press releases or front-line website material.[The NZTA referencing is substantially less helpful]

For all I know, the conclusions may be broadly correct. I wouldn’t be at all surprised if many drug users do believe silly things about their level of impairment. Before  the decades of advertising and enforcement, a lot of people believed silly things about the safety of drunk driving.  And the new TV ads are clever, even if they aren’t as good as the ‘ghost chips’ ad.  But the numbers used to advertise the campaign don’t mean what the people providing the money say they mean.  That’s not ok when it’s politicians or multinational companies, and it’s still not ok when the campaigners have good intentions. [Edited to add: I think this last sentence still stands, but should be directed at least equally at the NZTA].

[Update: Media links: TVNZ,  3 News, Stuff, NZ Herald, Radio NZ]

Keith Humphreys, an addiction researcher at Stanford, writes The newest Behavioral Risk Factors Surveillance System survey by the US CDC shows a substantially higher rate of binge drinking than in past surveys.  BRFSS is the world’s largest telephone survey, and in 2009 they started calling cellphones for the first time.

Cellphone users, and especially those who don’t have any landline phone, are a lot younger on average than the rest of the population.  That in itself need not be disastrous for surveys, since we know what proportion of the population is in each age group, and can rescale the numbers to remove the bias.  The problem is that cellphone users also are different in other ways that are harder to measure, as the CDC’s experience shows.

The papers are reporting an announcement of cheap DNA sequencing (or, as they like to put it ‘decoding’).  The new machine claims to be able to produce a human whole-genome sequence for USD1000, or about $1250 at current exchange rates.  This is, allegedly, a “milestone in bringing nearer the possibility of routinely sequencing a person’s entire DNA in order to identify, and possibly correct, genetic defects that could lead to disease or death”.  It remains to be seen what the accuracy and cost of the machine will be in practical use.  The $1000 doesn’t include any of the cost of blood samples and DNA extraction. It also doesn’t include the high administrative costs of clinical-quality sample tracking: in large-scale research you might be able to accept a 1 in 1000 chance of two samples being accidentally switched, but not in clinical medicine

Medical uses of sequencing are certainly closer as the cost falls, but cost is far from the only barrier.  At the moment, we have very little clue as to what DNA sequence variants are even predictive of disease risk. Even if disease risk could be predicted usefully, which is far from obvious for most diseases, that isn’t worth much clinically unless you can do something about it.  We’ve had the ability to measure thousands of genetic variants at much lower prices than $1000 for years, and haven’t been able to think of much worth doing with it clinically.

The main current applications of DNA sequencing in New Zealand are in the dairy industry, precisely because tiny improvements in prediction are much more valuable in agriculture (where you can manipulate breeding) than in medicine. The Livestock Improvement Corporation has been a leader in using DNA information to select animals for breeding, and a lot more cows than humans have had their genomes sequenced.

In the short term, both agricultural research and medical research should benefit from more-affordable genotyping. It may become possible to do some larger-scale genetic epidemiology in New Zealand. But you shouldn’t hold your breath for the NHI system to start paying for genome sequencing.

Stuff.co.nz (from Reuters) has a reasonably good article about a new research report in the journal JAMA on marijuana, tobacco, and lung function.  It’s also worth pointing out that, as usual, there is more information on line that the newspapers don’t tell you about: the abstract, and a video interview (ie, press release) with the main author.

The research report is from the CARDIA study, which is following up 5000 young adults in four US cities, primarily to look at cardiovascular disease risks.   This paper uses data on tobacco smoking, marijuana smoking, and lung function, and finds a decrease in lung function in people who took up tobacco smoking, but not in most of those who smoked pot.  The study is particularly interesting because the participants were recruited before they started smoking.

The researchers found that lung function of marijuana smokers doesn’t start going down until the cumulative exposure gets up to around 10 joint-years (ie, 1 joint/day for ten years, two/day  for five years, and so on), which is a pretty high level, reached by only a few of the CARDIA participants.  This is contrasted with tobacco, where negative effects start showing up at exposure levels that are quite common.  In fact, and this is whats getting the press, the average lung function is very slightly higher in pot smokers than non-smokers, though not by enough that you would notice without sensitive machinery and thousands of measurements.

Most of this is just that doses of marijuana are much, much lower.  In fact, the negative associations at 10 joint-years cumulative exposure to marijuana are pretty similar to those at 10 pack-years cumulative exposure to tobacco.  That is, the data are consistent with a single joint doing as much short-term lung damage  as a whole pack of cigarettes, and certainly indicate that a joint does more damage than a single cigarette.

The mysterious part is the slightly higher lung function among moderate-level pot smokers. The authors give a number of speculations.  My speculation would be that asthmatics, and other with sensitive airways, are slightly less likely to take up smoking pot.  I think this would actually be testable in the CARDIA data.  In any case, this is careful analysis of pretty high-quality data, done by people with no particular axe to grind, which makes a nice change.

Megan McArdle at The Atlantic critiques the recent online infographics trend:

…it’s time to get down to a war that really matters: the war on terrible, lying infographics, which have become endemic in the blogosphere, and constantly threaten to break out into epidemic or even pandemic status.

The reservoir of this disease of erroneous infographics is internet marketers who don’t care whether the information in their graphics is right … just so long as you link it.

Megan critiques a series of infographics and is well worth the read.

Hugo Chavez, the President of Venezuela, has publicly speculated that some secret US weapon might be responsible for several Latin American heads of government getting cancer recently, which he said was “difficult to explain using the law of probabilities.” This is a perfect example of a phenomenon that is all too familiar to public-health workers: the cancer cluster. According to the American Cancer Society, more than 1000 cancer clusters are reported to state public-health departments in the US each year. How many of them turn out to be real? Approximately none.

There are two statistical phenomena here. Firstly, although individual cancer subtypes may be rare, cancer as whole is more common than most people realize. Secondly, we are very good at seeing patterns, even patterns that aren’t there.

The picture, from David Spiegelhalter, shows four 9×9 squares. Three are coloured entirely at random; one has a pattern. Before going on, decide which one you think is non-random.

(more…)

Stuff.co.nz is reporting that twice as many Australians were arrested overseas last year as ten years ago.  A couple of minutes searching finds an Australian Social Trends report, from the Australian Bureau of Statistics, showing that, yes, trips overseas by Australians also doubled from 2000 to 2010.

Even better would be data on total time spent overseas by Aussies.  That must be available — they collect it when you re-enter the country — but I don’t want to spend more than ten minutes on this. It’s not my day job.

“What someone doesn’t want you to publish is journalism; all else is publicity.” Paul Fussell

Duncan Hedderley, in a comment on the ‘Overgeneralising again’ post, suggested that the problem with those articles was really advertising masquerading as research, rather than imprecise statistical reporting.

In fact, most of the stories we comment on are like that. It can be interesting to see if you can find any statistics in the typical news broadcast or newspaper that aren’t just publicity or commentary.  That is, the story contains a number which is an estimate or summary for a well-defined population, and either

We’ve commented before on the annoying tendency of newspapers to claim that self-selected website polls actually mean something.  The media usually refers to the results as coming from “an unscientific poll”, but a better term would be “a bogus poll”.  In the interests of openness, democracy, and giving you something to do over summer, we are conducting a bogus poll ourselves, to find out which terminology is better.

The NZ Herald online has had two stories in two days on a survey by Southern Cross Health Society.  The survey reported cancer as the number one health fear of Kiwis.  That would be the minority of Kiwis with private health insurance. Or, though the actual survey population isn’t stated anywhere, probably the smaller minority of Kiwis who are members of Southern Cross.

I don’t know whether the cancer fear finding generalises to the whole population, and neither do they, but I’m certain the screening results they report do not.  They say more than 80% of men aged 55-64, and 93% of men over 65 had a prostate cancer screening test within the past year.  Figures based on a national survey published in 2010 by the University of Otago, show that about 44% of men 60+ and 17% of men 40-60 had a PSA test in the the previous year (which includes diagnostic and post-treatment as well as screening tests).  Only 64% of men over 60 had ever had a PSA test of any sort and only 32% had ever had a screening PSA test as part of primary care.   It’s hardly surprising that people with private health insurance get more screening, though it’s still not completely clear to anyone except perhaps Paul Holmes whether the extra PSA screening is doing them any good.

The statistical message is simple: surveys only measure what they measure, not what you would like them to have measured. Get over it.