Posts from March 2017 (37)

Team Ratings for Round 5

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 34 matches played, 25 of which were correctly predicted, a success rate of 73.5%.
Here are the predictions for last week’s games.

Predictions for Round 5

Here are the predictions for Round 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 4

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 24 matches played, 11 of which were correctly predicted, a success rate of 45.8%.
Here are the predictions for last week’s games.

Predictions for Round 4

Here are the predictions for Round 4. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

There was a story on Stuff yesterday claiming that 85% of Kiwis brush their teeth too hard, based on a survey by a company that sells soft toothbrushes. The survey involved over 1000 people, and that’s about all we know, except that the reported rate of twice-per-day brushing was about 15 percentage points higher than in the 2009 NZ Oral Health Survey.

Mark Hanna tweeted about the press release by survey issue

I want to expand on this.  Why is research, especially survey research, different?  When David Fisher interviews people involved with gambling addiction, we don’t need anything more than his story. When Fletcher Building says they are cutting $150m from their profit estimates, we don’t need anything more than their press release. So why isn’t enough that this toothbrush company says they have a survey?

The issue is responsibility.  If an investigative journalist reports statements from people, it’s the journalist’s reputation that makes those reports credible. If there are anonymous sources, again it’s the reputation of the journalist and the newspaper that makes us believe the sources really exist and are their claims are credible.  When a company says it’s introducing a new product or is revising its income estimates, the company is the only authoritative source of information, and the claims are treated as mere claims by the company, not as facts.

With a survey press release, the journalist typically isn’t vouching for the correctness of the interpretation or the validity of the methodology; that’s not their expertise.  And we can’t tell from the toothbrush story whether it was a real survey, or a well-calibrated online panel, or whether it was just a bogus clicky poll on a website somewhere. There’s no attribution, and there’s no responsibility. Even so, the claims don’t get treated as mere advertising; they get reported in basically the same way as all research findings.

If we’re going to treat a survey of this sort as showing anything,  and if the journalist isn’t vouching for the information, the minimum standard is that we can find out what was done.  The company doesn’t need to nerd up its press release with details, but they can put them on a website somewhere — how they found people, what the response rate was, something about who they sampled, what the actual questions were. Or, if their survey was done by a reputable market research firm, tell us that, and at least we know someone who understands the issues is standing behind the claims.

There’s a story on Radio NZ Nine to Noon on a blood test for autism that is supposedly 96% accurate, and on what its implications might be for earlier specialised care of kids on the autism spectrum. As you’d expect from Radio NZ, the questions about the implications of a test were good.  The interview also raised the question of whether the test, developed on kids aged 3-10, would work equally well on younger children where the benefit is potentially greater.

What got missed out on a little was the meaning of “96% accurate”.  In the research study, 81 of 83 children on the autism spectrum and 73 of 76 neurotypical kids with no history of behavioral or neurologic abnormalities were correctly diagnosed, which is very impressive. But that’s not how the test would be used. In practice you might be screening the whole population, or screening kids with relatives on the autism spectrum, or diagnosing kids where ASD is suspected.

For whole-population screening, the 3.9% false-positive rate is more of a problem.  Based on current US statistics, roughly 1.5% of children are on the autism spectrum. So, of 1000 kids tested, about 15 would be correctly diagnosed and about 39 would be false positives. Now, it could be that the benefit to the 15 is much larger than the harm to the 39 and the test is worthwhile, but if we were going to have policy discussions about the test in that context, “96% accuracy” wouldn’t be a helpful way to describe it.

For children who have relatives with ASD you’d expect it to be harder to separate out ‘neurotypical’ from not. And that’s what the research shows:

The red and blue curves show the variation in blood test results for unrelated neurotypical and autism-spectrum children. The yellow shows the results for neurotypical siblings of children on the autism spectrum: there’s a lot less separation, and more tests will be wrong. Again, the test might be accurate enough to be useful, but “96% accuracy” is misleading.

Perhaps the way the test could be most helpful is as a step in diagnosis for children whose behaviour makes parents or doctors suspect ASD. There, the proportion on the autism spectrum might be more similar to the 50% in the study. However, we’ve currently got no idea how accurate the test is in such children; no-one has looked.

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday March 24 2017.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of March 18 – 24 2017 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

If you’d like to comment on or debate any of this week’s Stat of the Week nominations, please do so below!

• A guy tracked all the words his son learnt in the first 20 months, and made graphs.

• Based on survey data, Peter Beinart argues in the Atlantic “those who don’t regularly attend church are more likely to suffer from divorce, addiction, and financial distress.” Fred Clark (who does regularly attend church) has a different suggestion “It’s also entirely possible — and quite likely once you allow yourself to think about it — that all of these things make church-going more difficult and less likely.” 

• From the archive that noted.co.nz now has online, Mark Broatch interviews Eula Bliss about her book on vaccination.

• Apps that use your phone to monitor your health or nudge you towards better behaviour have obvious potential.  One of the first evaluations of an app baseed on Apple’s ResearchKit shows that the potential isn’t matched by actual. “only 131 participants took at least a week’s worth of surveys and a six-month milestone survey. That’s 1.7 percent”. Story at Ars Technica, research paper

• The new newsroom.co.nz has a good story on the history and uptake of a treatment for premature babies that came from New Zealand research on sheep.

• Mark Hanna has an interactive graphic comparing how often NZ Police use different tactical options for stopping people, by ethnicity.

• “This company wants to analyze your saliva — to try to predict when you’ll die” It probably won’t work and bring down the life insurance industry, but something will, eventually, unless regulations change. Also, ick.

• Does it matter how long you’ve already waited for a bus? In New York, not really.

• Nat Dudley talks about colour-blindness and accessibility of graphics.

• Harkanwal Singh talks about data journalism, for the Stats Department’s Ihaka Lectures.

PCSK9 inhibitors are one of the high-profile stories of genomics in medical research.  The gene’s function was previously unknown: it was identified as important for cholesterol metabolism by genetic studies.  People with mutations that increase the activity of the protein have high LDL cholesterol; people with mutations that destroy the activity have low LDL cholesterol. And, importantly, there’s at least one person walking around and alive and healthy with mutations breaking both her copies of the gene, so inhibiting it looked relatively safe.  It was an obvious target for drug development and a showcase for the benefits of large-scale genetic research.

Three drug companies have made injectable antibodies that block the activity of PCSK9 and dramatically lower LDL cholesterol. One dropped out last year because their drug got attacked by the patients’ immune systems.  We’re now seeing the first results of clinical trials looking at whether the LDL cholesterol reduction leads to fewer heart attacks.

From a New Zealand point of view, the results are mostly of theoretical interest. Pharmac isn’t likely to subsidise these treatments for large groups of people any time soon.  However, we do still care what the trials show, because they help answer some questions about cholesterol. The research paper is here; two good commentaries on it are here and here

Amgen’s drug, evolocumab, reduced LDL cholesterol by about two-thirds (from an average of 2.3 mmol/l to 0.78 mmol/l). The combined rate of heart attack, stroke, and death from heart disease was 20% lower; there was only a 15% reduction in the longer shopping list of bad events that the study put its money on for the primary analysis.

So, first, lowering LDL cholesterol by a different mechanism from statins has also resulted in lower heart attack rates.  That reinforces the evidence that LDL cholesterol really matters; it’s not just a marker like smoke from a fire.  Given the largely failed efforts to improve health with drugs that raise HDL cholesterol, this is good to know. Second, lowering LDL cholesterol this way seems fairly safe. There wasn’t any detectable harm (apart from the localised symptoms of the injections themselves). There could be rarer or more subtle effects, of course.

And finally, while the results are qualitatively positive, the actual scale of the benefit is a bit disappointing.  With an average 2.2 years of followup for 13784 people in the treatment group, about 200 heart attacks, strokes, or heart disease deaths were postponed or prevented.  At current US prices of $14,000 per year, that would cost over US$420 million.

So, two cheers for genomics in drug development.

The Herald story starts off

Commonly bought over-the-counter painkillers including ibuprofen have been linked to a significant increased risk of cardiac arrest.

The research paper is here (but paywalled).

First, it’s important to remember that “significant” in this context means “detectable” rather than “important.” The risk was higher by about 30%, but cardiac arrest is fairly rare.  With ten years of complete data from Denmark (about 5.5 million people) the researchers accumulated 30,000 cardiac arrests: that’s about five cases per ten thousand people per year.

As usual, this is observational data looking at correlations; the harmful effect, if it’s real, is too small to see reliably in clinical trials.  The researchers used a clever study design where they compared use of painkillers in a cardiac-arrest patient both with the same patient at times in the past and with different patients at the same time.  Differences between people that are constant over time (like smoking) will cancel out of the analysis; differences over time that are constant between people (like season) will also cancel out.  The design doesn’t cancel out non-constant factors like starting an exercise programme that leaves your muscles and joints sore.  It’s not unreasonable that a risk difference this small could be explained by confounding factors.

There’s something more important wrong with the story, though. You might wonder how people who have cardiac arrest get asked about their painkiller use. They didn’t; the study used prescription data.  For many of the painkillers, prescription is the only source; in particular, that’s the case for diclofenac (Voltaren), where the apparent risk increase in the study was a bit larger.

Ibuprofen, however, is available over the counter in Denmark, just as it is here. It’s available in fairly small packages, and is labelled for short-term use, just as it is here. Over-the-counter sale is what the story is basically about, but the study didn’t look at over-the-counter use at all.

From Stuff

3/2017: “New Zealand homeowners are being told to fix their interest rates now if they want to avoid a looming increase.”

Also from Stuff, all either headline or lead:

12/2016: Mortgage holders urged to fix as US interest rates rise

7/2016: Warning interest rates may not have much further to fall

3/2014: Time to fix loan on your house

1/2014: Rush to fix home loan rates before Reserve Bank acts

9/2013: “Economists say homeowners have officially missed the boat on locking in cheap fixed mortgage rates.”

4/2011: Now’s the time to fix mortgages: Tower

8/2009: Fix your mortgage before rates rise

There are good reasons to believe today’s story, but presumably there were for the past stories, too.