Q: So what’s with this green tea article at Stuff.co.nz?
A: It’s reporting an experimental study on weight loss using green tea extracts
Q: And did it work?
A: Yes, the treated subjects gained less weight than the untreated ones, by about 4g. (more…)
According to the New York Times, the US Preventive Services Taskforce is about to recommend against routine prostate-cancer screening, based on the results of randomized trials that didn’t show any decrease in overall deaths. We’ll come back and discuss the report when it actually appears, but this recommendation is going to be profoundly unpopular with certain groups, most dramatically with men who have had prostate cancers detected and removed.
One of the biggest problems in recommending against an existing screening method or treatment is all the complaints from people who think they have personal experience of its effectiveness (most of whom, I can safely predict, won’t actually bother reading the report before denouncing it). The idea that an individual can know that a treatment works, based just on personal experience, is a very powerful cognitive illusion. It can’t possibly be true — medical science would be so much simpler if it were — but the perception is unavoidable. For example, I had mild `walking’ pneumonia a few years ago, and got sicker over a period of weeks, then got treated, and suddenly started to feel better. It certainly felt as if the treatment worked, but when I later talked to some experts I found out that there is very little reason to believe that antibiotics are helpful for the sort of illness I had, and suddenly starting to feel better is exactly what happens when your body finally gets on top of an infection. Even so, if you hooked me up to a polygraph you would find I still believe the treatment worked for me.
Cognitive illusions are much like optical illusions: these lines don’t look straight, and you can’t learn to see them as straight, no matter how much optics you study. You just have to decide who you are going to believe — the evidence, or your own lying eyes?
Scientific American’s blog is having a series of posts on how clinical trials for new medicines work (the author describes the series as ‘medicine for muggles’).
New vaccines only get approved after randomized controlled trials showing that they are beneficial, but if you want to estimate the benefit of expanding vaccination to a new group of people it’s hard to do a randomized trial. Just comparing vaccinated and unvaccinated people doesn’t help, since there are many reasons why these groups are different, and the comparison gives completely unreasonable estimates. There’s a new study from Canada, reported by Reuters, that takes advantage of a ‘natural experiment’ to estimate the benefit of vaccinating children aged 2-4.
In the US, the guidelines on vaccination changed in 2006 to include kids in this age range, in Canada the guidelines didn’t change until last year. This allowed the researchers to compare hospital emergency room visits in Montreal and Boston and estimate the impact of the change. Just looking at US hospitals wouldn’t be enough, since there is a lot of year-to-year variation in the severity of the current flu strains, and just comparing the US to Canada wouldn’t work, since there are a whole lot of differences between the countries (starting with a national health insurance system). But looking at how the US:Canada difference changed from before 2006 to after 2006 gives a reasonable estimate of the effect of vaccination.
Looking at over 100,000 emergency-room visits for flu-like illness, researchers found a 34% decrease in risk for the 2-4 year age group affected by the change in guidelines. This wasn’t just for kids who were actually vaccinated — it also includes the reduction in risk from having your playmates vaccinated. There was a smaller reduction in risk, 10-20%, for older children — either because the additional reminders made them more likely to get vaccinated, or because they were less likely to catch flu from younger siblings.
Natural experiments get used a lot in economics. In medicine, we tend to prefer real experiments, but sometimes these are impractical or unethical, and natural experiments are the best we can do.
The factoid of dramatically increasing cannabis potency has popped up again, with a claim that cannabis used to be 1-2% THC and is now up to 33%. The most comprehensive and consistent data on cannabis potency come from a long-term project at the University of Mississippi. Their 2010 paper is based on analysis of 46,000 confiscated samples from 1993 to 2008. Over this time period, the percentage of THC in marijuana (leaves and buds with seeds) increased from about 3.5% to about 6%. The percentage in sinsemilla (buds without seeds) increased from about 6% to about 11%. Since the more-recent samples were more likely to be sinsemilla, the percentage over all confiscated samples increased a bit more, from about 3.5% to about 9%. A small fraction of the samples had much higher concentrations, but this fraction didn’t change much over time. So, yes, the average used to be about 3% in 1993 and may have been as low as 2% in earlier decades, and, yes, the concentration is now ‘up to‘ 33%, but the trend is nothing like as strong as that suggests. A New Zealand paper , by ESR researchers (who are hardly pot-sympathising hippies), says that there was no real change in THC concentration in cannabis plant material from 1976 to 1996, and the concentration in cannabis oil actually fell.
The Southland Times article also reports a claim that 90% of first-term methamphetamine users continue to use the drug. If this just means that 90% of them go on to have a second dose at some time it might well be true, but if it is implying long-term addiction the figure seems implausible. It’s certainly not what is found in other countries. For example, the most recent results from the US National Survey on Drug Use and Health (NSDUH) estimate that 364000 people in the US had dependence/abuse of illegal stimulants in 2010. If we assume that all of these were methamphetamine, and that the other illegal stimulants didn’t cause any dependence/abuse problems, that’s still only 20% of the estimated 1.8 million people who first tried methamphetamine in the period 2002-2010. In fact, since NSDUH has a nice online table generator we can do a more specialized query and find out that an estimated 118000 people currently had dependence on stimulants out of the estimated 10 million people who had ever tried methamphetamine. That’s more like 1% than 90%. Amphetamines are clearly something you want to stay well away from, but there’s no way that they addict 90% of the people who try them. In any case, if we believe the drug warriors, New Zealand’s P epidemic has already been solved by banning pseudoephedrine without a prescription.
I’m all for getting teenagers to appreciate the risks of drug use, but we need to remember teenagers can use Google too.
Last week’s Stat of the Week nomination was a story on the “J-curve” for disease risk and alcohol consumption. Yet another research paper, this time from the Nurses’ Health Study, had found that people who drink small amounts of alcohol regularly are healthier than those who drink none and those who drink larger amounts. This sort of result is unpopular with doctors, as the Herald story reported, and for good reasons, but that doesn’t mean it’s untrue. On the other hand, the fact that it’s true doesn’t mean that it’s news.
The obvious difficulty in comparing drinkers to non-drinkers is that some of the strictest non-drinkers are actually ex-drinkers, people who you would expect to be in worse health. Since epidemiologists are not completely stupid, they know about this problem and many studies have addressed it. Excluding ex-drinkers doesn’t make the effect go away, waiting for a long time between the drinking assessment and the health assessment (as in this paper) doesn’t make it go away, and splitting up light drinking into finer categories shows that there is lower risk for people who drink occasionally than for those who regularly drink a small amount (again, as in this paper). For some of the claimed benefit there are even plausible mechanisms (eg, alcohol consumption does definitely raise HDL cholesterol levels in short-term experimental studies). This is just observational research, so the results could be just as wrong as the apparent protective effect of beta-carotene in cancer, or of raising HDL cholesterol with niacin in heart disease, which fell apart when subjected to randomised trials, but it’s carefully-done observational research.
As doctors will tell you, the problem with announcing a health benefit of moderate alcohol consumption is that most people interpret “moderate” to mean “a bit more than I currently drink”. As a scientific result, it’s fine; as a public-health intervention, it’s badly off-target. The American Heart Association guidelines on alcohol and heart disease, for example, basically say that regular consumption of small amounts of alcohol probably is protective against heart disease, but that you shouldn’t go around advocating it.
The problem for medical researchers is that funding bodies and universities (and their own egos) want press coverage of research results, but that this sort of marketing of incremental medical research as if it was ground-breaking health advice is unhelpful to the public. It’s very rare that you should change your behavior based on the results of a single medical study, but that’s the model that a lot of medical reporting is based around.
In this week’s Stat of the Week the misinterpretation is not primarily the fault of the individual media outlet, since it was present in the original source. Still, if a press release or a wire service story told you that the Wallabies had a new training regimen that would improve their game without making them fitter, faster, tougher in the scrum, more accurate with kicking, or better at putting in the elbow, you’d ask questions. We’d like to see science journalism eventually get up to the standard of sports journalism.
The Herald reported “a new study suggests [TV’s] damaging effects may even rank alongside those from smoking and obesity”. If you look at the British Journal of Sports Medicine, that’s what the authors actually say. They go on to say “TV viewing time may have adverse health consequences that rival those of lack of physical activity, obesity and smoking; every single hour of TV viewed may shorten life by as much as 22 min”. The implication that TV has an effect separate from physical activity and obesity, just as it is separate from the effect of smoking, is reinforced when they say that the associations were adjusted for a whole bunch of cardiovascular risk factors: cholesterol, blood pressure, age, gender, weight, blood glucose, etc. The implied claim is that TV kills in a way that isn’t explained by any of these risk factors: it’s not that TV-watching uses up fewer calories, or that you are more likely to snack while watching. Perhaps the mechanism is that watching too much TV makes you believe all the health-related advertising and medical news? (more…)
The NZ Herald reports (from a Cancer UK press release): A single genetic fault in a gene that normally helps the body to repair its DNA increases a woman’s risk of ovarian cancer six-fold, a study has found. That sounds as if it might be useful as a way of detecting women at high risk, until you look at the numbers in more detail.
In the UK, where the study was conducted, about 6500 women per year are diagnosed with ovarian cancer, and 40-50 of these women will have the genetic fault. New Zealand has about 15 times fewer people than the UK, so that would be about 3 women per year in the whole country who develop ovarian cancer because of this genetic variant.
Q: So, have you seen the article about Vitamin D and diabetes?
A: Of course. The tireless staff of StatsChat read even West Island newspapers. It’s a good report, too.
Q: What did the researchers do?
A: They studied 5200 people without diabetes, following them up for five years. 199 of them developed diabetes. The people who ended up with diabetes started off with lower vitamin D levels in their blood.
Q: Where did you get those details?
A: The abstract for the study publication (you can also get the full text there free if you’re at a university or if you wait until next year).
Q: Isn’t it annoying that newspaper websites don’t provide any links to that sort of information?
A: It’s like you’re reading my mind.
Q: One of the study authors is quoted as saying “”It’s hard to underestimate how important this might be.” What do you think?
A: I think he meant “overestimate”.
Q: So, how important is this finding?
A: If it really is an effect of vitamin D, it would be really important. A simple supplement would be able to dramatically reduce the risk of diabetes.
Q: How can we tell?
A: Someone needs to do a randomized trial, where half the participants get vitamin D and half get a dummy pill. If the effect is real, fewer people getting vitamin D will end up with diabetes.
Q: That sounds like a good idea. Is someone doing a trial?
A: Yes, Professor Peter Ebeling, of the the University of Melbourne.
Q: Is there some useful website where I can find more information about the trial?
A: Indeed.
Q: Will it work?
A: No.
Q: Are you sure?
A: No, that’s why we need the trial. But it’s a trial of vitamin supplementation, which almost always has disappointing results, and it’s a trial in adult-onset diabetes, which almost always has disappointing results.
Stats.org has published an excellent article on the research surrounding risk factors for Sudden Infant Death Syndrome (SIDS), in particular breastfeeding:
Let there be no doubt: not breastfeeding and SIDS are correlated. The problem is that breastfeeding is correlated with many other factors as well, any of which could be the “cause” (or “causes”) behind an increased SIDS rate among people who use formula instead of mothers’ milk. These include a variety of social and cultural differences, differences in care, differences in other feeding patterns, differences in sleeping patterns, differences in genetic makeup, differences in home environment, differences in medical care, etc. The question is whether the evidence points to breastfeeding (or mother’s milk) as a preventative factor by itself and independent of all the other factors with which breastfeeding tends to go hand in hand.
It continues to unravel the statistics and evidence and concludes by saying:
Without the science, the claims of cost due to not breastfeeding – 447 babies and almost 5 billion dollars in economic loss — are like an empty bottle: wanting for real substance.