Stats Chat

Team Ratings for Round 3

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 14 matches played, 9 of which were correctly predicted, a success rate of 64.3%.
Here are the predictions for last week’s games.

Predictions for Round 3

Here are the predictions for Round 3. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Q: Did you see hydroxychloroquine is back?

A: No

Q: In the Herald (blamed on news.com.au) ‘Hydroxychloroquine: Divisive drug may hold secrets to stopping Covid-19‘ and ‘Covid 19 coronavirus: Hydroxychloroquine: The drug that could be our saviour‘. What’s the news?

A: There’s a new Australian trial in healthcare workers, to see if taking the drug before you get exposed will protect you  from infection — previously there had only been evidence that it doesn’t help  if you take it when you’re already sick.

Q:  And what were the results?

A: There aren’t any results. There won’t be any results for quite a while.

Q: When?

A: The entry at the Clinical Trials Registry says they plan to finish taking measurements at the end of the year, and the story says the results will be in by January.  Though  the Trials  Registry also  says they plan to recruit 2250 people and follow them for four months, and the story says they have ‘roughly 200’ people now. So it’s not completely clear.

Q: Will it work?

A: We don’t know. That’s the point of the trial.   We know it’s nowhere near 100%  effective,  because people taking hydroxychloroquine for auto-immune diseases have  ended up with COVID, but it’s possible that it provides some useful level of protection. It’s also very possible that it doesn’t.

Q: And healthcare workers are at high risk, so it would be most useful for them?

A: Yes, and healthcare workers are already trying to do all the other protective things, and they are still at high risk, so the drug might be a useful addition even if it’s only moderately effective

Q: And for the rest of us?

A: It’s unlikely to be as safe or effective as masks.

Q: If it does work, it will be pity that the politicisation has slowed it down

A: Well and the fact that it doesn’t work after you get sick. But yes, that’s one of the points the story makes, quoting both the lead researcher and a study participant

Q: This is the story that’s illustrated with a picture of Donald Trump?

A:  <sigh>

Q: Apart from the headline and picture, the story is ok?

A: Well, later on, one of the researchers says

“For example if there was a case in a meatworks or an aged care, you’d go there and give the drug to all the residents or workers to try to prevent them getting Covid-19,” he said.

Q: But how is that before they’re exposed? If there’s a diagnosed case, they’ve already exposed people and they will be isolated in the future and not expose anyone else. It’s the people who are already exposed that are the problem.

A: I hope he’s just saying there’s a potential for using it as post-exposure prophylaxis in the future, after a different trial

Q: That … could have been clearer.

A:  And the biggest problems for healthcare workers in the current Australian outbreak seem to have been a shortage of protective equipment or poor ventilation, so you’d hope irresponsible news headlines about a miracle cure wouldn’t distract from that.

Moderna (big long PDF) and Pfizer (big long  PDF) have released their clinical trial protocols.  These specify how  to run the trial: the statistical design, and also a lot of other things  like how  they collect adverse events. There’s a story at Reuters and at Buzzfeed. I’m not the right person to  comment about the detailed collection of safety data, except to say that they do describe how they do it,  so you can ask your favourite immunologist or virologist for an opinion.

The statistical design of early stopping is reasonable in both trials, as you’d hope. Both trials will end up declaring a positive result with an estimated vaccine efficacy  greater than 0.5, and with  0.3 efficacy outside the margin of error.   If the true efficacy is 0.6 they have a  90% chance of  declaring a positive result.

They vary in how they place their bets for the case where the vaccine is more effective than that — and it might be, who  knows?.  If the vaccine is actually 90% effective, you don’t need to test it on as many people, and the urgency of knowing it works is greater. The trials are designed so  they can stop early, but still preserve their overall standards for statistical evidence.

The tradeoff for stopping  earlier is that you have less detailed  information: you  know the vaccine is  good, but you have less precise estimates of how good, and you have somewhat less safety information because you’ve vaccinated fewer people or followed them for less time.  There is an important and nerdy statistical literature on the  pros and cons of basically any possible set of guidelines for stopping early. If you’re interested, this is a good starting place.

Moderna designed their trial more or less the way I would have done, with the so-called O’Brien-Fleming guidelines, which are standard and pretty conservative about early stopping. Pfizer are less conservative: if the vaccine is very effective, they have more chance of  stopping early.   But while they are less conservative, the design is still well in the  range of standard custom and practice.

We still don’t have any idea what the results will be at the early analyses, either for effectiveness or safety.  We don’t know what the US  government will decide to do with the results. But we do know that if the trials follow their protocols, and if the trials stop early, and if there aren’t  prohibitive safety concerns, then we will have good  evidence that  the vaccine works.   We  also will be able to tell if  the companies or  the FDA or anyone changes the standards of evidence after seeing the results.

The Herald

Researchers from the Australian National University have now developed a new test which picks up previous Covid-19 infection in a patient’s blood

The study indicates eight in 3000 healthy and previously undiagnosed Australians had likely been infected with the virus.

“This suggests that instead of 11,000 cases we know about from nasal swab testing, about 70,000 people had been exposed overall,” Associate Professor Ian Cockburn said.

We had a few of these ‘seroprevalence’ studies a while back. If you’re trying to estimate a proportion as low as 8 in 3000 from a sample, you need a representative sample and you need a test with a false-positive rate that you know is much lower than 8 in 3000.

Let’s look at the preprint:

You don’t need to download the PDF,  just skimming the abstract will tell you how they got the 3000 people, and what the uncertainty is:

 We used this assay to assess the frequency of virus-specific antibodies in a cohort of elective surgery patients in Australia and estimated seroprevalence in Australia to be 0.28% (0 to 0.72%)

Emphasis added: the uncertainty interval goes all the way down to  zero.   In contrast to some of the earlier seroprevalence studies, they seem to have done the analysis right, but I’m not convinced that people getting  surgery are a representative sample — they certainly aren’t a random sample.

If you do click through the PDF, the Discussion section says it even more clearly

Here we report results from the first large scale seroprevalence survey in Australia. We estimate a seroprevalence of 0.28%, which–given a population estimate for Australia of 25.50 million individuals — equates to 71,400 infections(95% CI: 0 to 181,050).

It’s actually pretty impressive  that the test is as good as it is, but it’s still not really up to the challenge of providing reliable evidence on the number of people exposed to Covid in Australia.

• Good piece at Stuff, from The Conversation about COVID statistics
• “The BSA said the decision highlights the importance of data literacy, particularly in a news and current affairs context.” Broadcasting Standards Authority decision, on a complaint about Mike Hosking.
• Two StatsChat-relevant new books: “How to Make the World Add Up” by Tim Harford, and “Calling Bullshit” by Carl Bergstrom and Jevin West
• “Forecast models”…are typically designed with a single goal in mind: to make a specific, quantitative prediction about an event that will be observed in the future….Other infectious-disease “scenario models” are designed to explore multiple “what if” hypothetical futures”.  From the Washington Post
• A story at the Herald, from the Daily Telegraph: low-level exposure to coronavirus through masks  could be giving people immunity. It could be, but there’s little  to no evidence that it actually is.  And we know it isn’t in New Zealand, because there’s almost no coronavirus here to have low-level exposure to, a point that might have been worth mentioning.

Some graphs of the stringency of NZ covid restrictions (from the Oxford Coronavirus Government Response Tracker) because it’s easy to forget what the  rest of the world is like if you don’t  talk to them regularly.

Here is  New Zealand and some countries we get compared to:

That’s not an ideal graph because it’s based on the maximum severity anywhere in the country,  with just a small offset for not being national. Here I’ve tried to work it out as  if Auckland and the Rest of NZ were separate countries. It’s still  not perfect, because other countries also have sub-national variation.

And there is the same comparison for another relevant group of countries

As a bit of light relief from Covid, let’s look at this story at Stuff. It’s mostly about maternity wear (correctly disparaging the maternal-industrial complex), but there’s this

There’s having a face. Yes, you should be ashamed of your natural face. Luckily, there’s all manner of products to correct it, and makeup to cover it up. You’ll need both, to adequately conceal that shameful natural face of yours. It’ll cost the average woman about $400,000 over her lifetime.

If that were true, it would  leave avocado toast in the dust.  Where  does this figure come from?

The story doesn’t say, but there’s a similar figure that’s quite popular, eg at Buzzfeed,

A new survey from Skin Store of more than 3,000 women found that the average woman in the US spends around $300,000 on makeup in her lifetime.

The increase from $300k to $400k looks like it could easily be a currency conversion.  Buzzfeed (of course) doesn’t link to information about  the actual survey, but it is available.

The first thing to notice is

When you take into consideration how much the daily face costs of our New York women, they can spend up to $300,000 per lifetime on skincare products and cosmetics.

So the figure has been misquoted in translation, moving from New York women to US women to NZ women.  It’s probably not true even there — Skin Store don’t give any information about methodology.

We can see how badly off the number must be with a bit of simple arithmetic and a few Google queries. Skin Store says they surveyed women aged 16 to 75. There are about 100 million women aged 18 and over in the US. That’s not quite  the right age range, but it will do as an approximation.  Multiplying by US$300k per lifetime gives US$30 thousand billion.  Dividing by the 60-year ‘make-up lifetime’ gives half a trillion per year.  Total US GDP  is about $20 trillion.  The claim is that more than 2% of total US GDP goes on face-care products. For comparison, the US spends about 0.75 trillion on primary and secondary education, 1.7 trillion on all food.

The Bureau of Labor Statistics, who actually care about getting this sort of thing roughly right, estimate an average annual expenditure of less than US$800 per ‘consumer unit’ on all ‘Personal care products and services’. A ‘consumer unit’ is roughly what normal people would call a household or  family, and $800/year (which includes a lot more than just makeup, and not just women) comes to $48,000 per 60 years.  The Skin Store number looks to be off by around a factor of ten.   Just making things up would be more accurate than that.

Team Ratings for The Final

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 21 matches played, 18 of which were correctly predicted, a success rate of 85.7%.
Here are the predictions for last week’s games.

Predictions for The Final

Here are the predictions for The Final. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 21

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 120 matches played, 79 of which were correctly predicted, a success rate of 65.8%.
Here are the predictions for last week’s games.

Predictions for Round 21

Here are the predictions for Round 21. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 19

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 144 matches played, 100 of which were correctly predicted, a success rate of 69.4%.
Here are the predictions for last week’s games.

Predictions for Round 19

Here are the predictions for Round 19. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.