Posts filed under Risk (138)

November 16, 2014

John Oliver on the lottery

When statisticians get quoted on the lottery it’s pretty boring, even if we can stop ourselves mentioning the Optional Stopping Theorem.

This week, though, John Oliver took on the US state lotteries: “..,more than Americans spent on movie tickets, music, porn, the NFL, Major League Baseball, and video games combined. “

(you might also look at David Fisher’s Herald stories on the lottery)

November 7, 2014

What overdiagnosis looks like

An article in the New England Journal of Medicine talks about screening for thyroid cancer in South Korea. There has been a massive increase in diagnosis, mostly of very small tumours that are probably harmless — there was been no change in the thyroid cancer deaths.

thyroid

As the authors say:

Thyroid-cancer surgery has substantial consequences for patients. Most must receive lifelong thyroid-replacement therapy, and a few have complications from the procedure. An analysis of insurance claims for more than 15,000 Koreans who underwent surgery showed that 11% had hypoparathyroidism and 2% had vocal-cord paralysis.

 

November 3, 2014

It’s warmer out there

Following a discussion on Twitter this morning, I thought I’d write again about increasing global temperatures, and also about the types of probability statements.

The Berkeley Earth Surface Temperature project is the most straightforward source for conclusions about warming in the recent past. The project was founded by Richard Muller, a physicist who was concerned about the treatment of the raw temperature measurements in some climate projections. At one point, there was a valid concern that the increasing average temperatures could be some sort of statistical artefact based on city growth (‘urban heat island’) or on the different spatial distribution and accuracy of recent and older monitors. This turned out not to be the case. Temperatures are increasing, systematically.  The Berkeley Earth estimate agrees very well with the NASA, NOAA, and Hadley/CRU estimates for recent decades

best

The grey band around the curve is also important. This is the random error. There basically isn’t any.  To be precise, for recent years, the difference between current and average temperatures is 20 to 40 times the uncertainty — compare this to the 5σ used in particle physics.

What there is uncertainty about is the future (where prediction is hard), and the causal processes involved. That’s not to say it’s a complete free-for-all. The broad global trends fit very well to a simple model based on CO2 concentration plus the cooling effects of major volcanic eruptions, but the detail is hard to predict.

Berkeley Earth has a page comparing reconstructions of  temperatures with actual data for many climate models.  The models in the last major IPCC assessment report show a fairly wide band of prediction uncertainty — implying that future temperatures are more uncertain than current temperatures. The lines still all go up, but by varying amounts.

best-gcm

 

The same page has a detailed comparison of the regional accuracy of the models used in the new IPCC report. The overall trend is clear, but none of the models is uniformly accurate. That’s where the uncertainty comes from in the IPCC statements.

The earth has warmed, and as the oceans catch up there will be sea levels rises. That’s current data, without any forecasting.  There’s basically no uncertainty there.

It’s extremely likely that the warming will continue, and very likely that it is predominantly due to human-driven emissions of greenhouses gases.

We don’t know accurately how much warming there will be, or exactly how it will be distributed.  That’s not an argument against acting. The short-term and medium-term harm of climate changes increases faster than linearly with the temperature (4 degrees is much worse than 2 degrees, not twice as bad), which means the expected benefit of doing something to fix it is greater than if we had the same average prediction with zero uncertainty.

October 30, 2014

Cocoa puff

Both Stuff and the Herald have stories about the recent cocoa flavanols research (the Herald got theirs from the Independent).

Stuff’s story starts out

Remember to eat chocolate because it might just save your memory. This is the message of a new study, by Columbia University Medical Centre.

 

Sixteen paragraphs later, though, it turns out this isn’t the message

“The supplement used in this study was specially formulated from cocoa beans, so people shouldn’t take this as a sign to stock up on chocolate bars,” said Dr Simon Ridley, Head of Research at Alzheimer’s Research UK.

 

There’s a lot of variation in flavanol concentrations even in dark chocolate, but 900mg of flavanols would be somewhere between 150g and 1kg of dark chocolate per day.  Ordinary cocoa powder is also not going to provide 900mg at any reasonable consumption level.

The Herald story is much less over the top. They also quote in more detail the cautious expert comments and give less space to the positive ones. For example, that the study was very small and very short, and the improvement in memory was just in one measure of speed of very-short-term recall from a visual prompt, or that this measure was chosen because they expected it to be affected by cocoa rather than because of its relevance to everyday life. There was another memory test in the study, arguably a more relevant one, which was not expected to improve and didn’t.

Neither story mentions that the randomised trial also evaluated an exercise program that the researchers expected to be effective but wasn’t. Taking that into account, the statistical evidence for the effect of flavanols is not all that strong.

October 28, 2014

Absolute, relative, correlation, cause

The conclusions of a recent research paper

Delivery by [caesarean section] is associated with a modest increased odds of [autism], and possibly ADHD, when compared to vaginal delivery. Although the effect may be due to residual confounding, the current and accelerating rate of[caesarean section] implies that even a small increase in the odds of disorders, such as [autism] or ADHD, may have a large impact on the society as a whole. This warrants further investigation.

The Herald

Babies born through Caesarean section are more likely to develop autism, a new study says.

Academics warn the increasingly popular C-section deliveries heighten the risk of the disorder by 23 per cent.

There’s a fairly clear difference in language: the news story is fairly clearly implying that caesarean sections cause autism; the research paper is being scrupulously careful not to say that.

Using a relative risk is convenient in technical communication, but in non-technical communication makes the impact seem greater than it really is. The US Centers for Disease Control estimate a risk of 1 in 68 for autism spectrum disorder (there aren’t systematic NZ data).  If the correlation with C-section really is causal, we’re talking about roughly 14 kids with autism spectrum disorders per 1000 without a C-section and about 17 per 1000 with a C-section. The absolute risk increase, if it’s real, is about 3 cases per 1000 C-sections.

It’s also important to be clear that this correlation cannot explain much of the recent increases in autism. A relative risk of 1.23 means that if we went from no C-sections to 100% C-sections there would be a 23% increase in autism spectrum disorder. The observed increase is about five times that, and since  C-sections have only increased about 10 percentage points, not 100 percentage points, the observed increase in autism is about 50 times what this correlation could explain.

There are (I’m told by people who know the issues) good reasons to think there are too many C-sections.  This probably won’t be one of the most important ones.

 

October 22, 2014

Screening the elderly

I’ve seen two proposals recently for population screening of older people. They’re probably both not good ideas, but for different reasons.

We had a Stat of the Week nomination for a proposal to screen people over 65 for depression at ordinary GP visits, to prevent suicide. The proposal was based on the fact that 70% of the suicides were in people who had visited a GP within the past month.  If the average person over 65 visits a GP less than about 8.5 times a year, this means those visiting their GP are at higher risk.  However, the risk is still very small: 225 over 5.5 years is 41/year, 70% of that is 29/year.

To identify those 29, it would be necessary to administer the screening question to a lot of people, at least hundreds of thousands. That in itself is costly; more importantly, since the questionnaire will not be perfectly accurate there will be  tens of thousands of positive results. For example, a US randomised trial of depression screening in people over 60 recruited 600 participants from 9000 people screened. In the ‘usual care’ half of the trial there were 3 completed suicides over the next two years; in those receiving more intensive and focused help with depression there were 2. The trial suggests that screening and intensive intervention does help with symptoms of major depression (probably at substantial cost), but it’s not likely to be a feasible intervention to prevent suicide.

 

The other proposal is from the UK, where GPs will be financially rewarded for dementia diagnoses. In contrast to depression, dementia is pretty much untreatable. There’s nothing that modifies the course of the disease, and even the symptomatic treatments are of very marginal benefit.

The rationale for the proposal is that early diagnosis gives patients and their families more time to think about options and strategies. That could be of some benefit, at least in the subset of people with dementia who are able and willing to talk about it, but similar advance planning could be done — and perhaps better — without waiting for a diagnosis.

Diagnosis isn’t like treatment. As a British GP and blogger, Martin Brunet, points out

We are used to being paid for things of course, like asthma reviews and statin prescribing, and we are well aware of the problems this causes – but at least patients can opt out if they don’t like it.

They can refuse to attend a review, decline our offer of a statin or politely take the pill packet and store it unopened in the kitchen cupboard. They cannot opt out of a diagnosis.

 

September 26, 2014

Screening is harder than that

From the Herald

Calcium in the blood could provide an early warning of certain cancers, especially in men, research has shown.

Even slightly raised blood levels of calcium in men was associated with an increased risk of cancer diagnosis within one year.

The discovery, reported in the British Journal of Cancer, raises the prospect of a simple blood test to aid the early detection of cancer in high risk patients.

In fact, from the abstract of the research paper, 3% of people had high blood levels of calcium, and among those,  11.5% of the men developed cancer within a year. That’s really not strong enough prediction to be useful for early detection of cancer. For every thousand men tested you would find three cancer cases, and 27 false positives. What the research paper actually says under “Implications for clinical practice” is

“This study should help GPs investigate hypercalcaemia appropriately.”

That is, if a GP happens to measure blood calcium for some reason and notices that it’s abnormally high, cancer is one explanation worth checking out.

The overstatement is from a Bristol University press release, with the lead

High levels of calcium in blood, a condition known as hypercalcaemia, can be used by GPs as an early indication of certain types of cancer, according to a study by researchers from the universities of Bristol and Exeter.

and later on an explanation of why they are pushing this angle

The research is part of the Discovery Programme which aims to transform the diagnosis of cancer and prevent hundreds of unnecessary deaths each year. In partnership with NHS trusts and six Universities, a group of the UK’s leading researchers into primary care cancer diagnostics are working together in a five year programme.

While the story isn’t the Herald’s fault, using a photo of a man drinking a glass of milk is. The story isn’t about dietary calcium being bad, it’s about changes in the internal regulation of calcium levels in the blood, a completely different issue. Milk has nothing to do with it.

September 10, 2014

Cannabis graduation exaggeration

3News

Teenagers who use cannabis daily are seven times more likely to attempt suicide and 60 percent less likely to complete high school than those who don’t, latest research shows.

Me (via Science Media Center)

“The associations in the paper are summarised by estimated odds ratios comparing non-users to those who used cannabis daily. This can easily be misleading to non-specialists in two ways. Firstly, nearly all the statistical evidence comes from the roughly 1000 participants who used cannabis less than daily, not the roughly 50 daily users — the estimates for daily users are an extrapolation.

“Secondly, odds ratios are hard to interpret.  For example, the odds ratio of 0.37 for high-school graduation could easily be misinterpreted as a 0.37 times lower rate of graduation in very heavy cannabis users. In fact, if the overall graduation rate matched the New Zealand rate of 75%, the rate in very heavy cannabis users would be 53%, and the rate in those who used cannabis more than monthly but less than weekly would be 65%.

That is, the estimated rate of completing high school is not 60% lower, it’s about 20% lower.  This is before you worry  about the extrapolation from moderate to heavy users and the causality question. The 60% figure is unambiguously wrong. It isn’t even what the paper claims.  It’s an easy mistake to make, though the researchers should have done more to prevent it, and that’s why it was part of my comments last week.

You can read all the Science Media Centre commentary here.

 

[Update: The erroneous ‘60% less likely to complete high school’ statement is in the journal press release. That’s unprofessional at best.]

(I could also be picky and point out 3News have the journal wrong: The Lancet Psychiatry, which started this year, is not the same as The Lancet, founded in 1823)

August 15, 2014

Cancer statistics done right

I’ve mentioned a number of times that statistics on cancer survival are often unreliable for the conclusion people want to draw, and that you need to look at cancer mortality.  Today’s story in Stuff is about Otago research that does it right:

The report found for 11-year timeframe, cancer-specific death rates decreased in both countries and cancer mortality fell in both countries. But there was no change in the difference between the death rates New Zealand and Australia, which remained remained 10 per cent higher in New Zealand.

That is, they didn’t look at survival after diagnosis, they looked at the rate of deaths. They also looked at the rate of cancer diagnoses

“The higher mortality from all cancers combined cannot be attributed to higher incidence rates, and this suggests that overall patient survival is lower in New Zealand,” Skegg said.

That’s not quite as solid a conclusion — it’s conceivable that New Zealand really has higher incidence, but Australia compensates by over-diagnosing tumours that wouldn’t ever cause a problem — but it would be a stretch to have that happen over all types of cancer combined, as they observed.

 

August 14, 2014

Breast cancer risk and exercise

Stuff has a story from the LA Times about exercise and breast cancer risk.  There’s a new research paper based on a large French population study, where women who ended up having a breast cancer diagnosis were less likely to have exercised regularly for the past five year period.  This is just observational correlation, and although it’s a big study, with 2000 breast cancer cases in over 50000 women, the evidence is not all that strong (the uncertainty range around the 10% risk reduction given in the paper goes from an 18% reduction down to a 1% reduction).  Given that,  I’m a bit unhappy with the strength of the language in the story:

For women past childbearing age, a new study finds that a modest amount of exercise — four hours a week of walking or more intensive physical activity such as cycling for just two hours a week — drives down breast cancer risk by roughly 10 per cent.

There’s a more dramatically wrong numerical issue towards the end of the story, though:

The medications tamoxifen and raloxifene can also drive down the risk of breast cancer in those at higher than average risk. They come with side effects such as an increased risk of deep-vein thrombosis or pulmonary embolism, and their powers of risk reduction are actually pretty modest: If 1000 women took either tamoxifen or raloxifene for five years, eight breast cancers would be prevented.

By comparison, regular physical activity is powerful.

Using relative risk reduction for the (potential) benefits of exercise and absolute risk reduction for the benefits of the drugs is misleading. Using the breast cancer risk assessment tool from the National Cancer Institute, the five-year breast cancer risk for a typical 60 year old is perhaps 2%. That agrees with the study’s 2000 cases in 52000 women followed for at least nine years.  If 1000 women with that level of risk took up regular exercise for five years, and if the benefits were real,  two breast cancers would be prevented.

Exercise is much less powerful than the drugs, but it’s cheap, doesn’t require a doctor’s prescription, and the side-effects on other diseases are beneficial, not harmful.