Stats Chat

Team Ratings for Round 21

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 120 matches played, 79 of which were correctly predicted, a success rate of 65.8%.
Here are the predictions for last week’s games.

Predictions for Round 21

Here are the predictions for Round 21. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 19

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 144 matches played, 100 of which were correctly predicted, a success rate of 69.4%.
Here are the predictions for last week’s games.

Predictions for Round 19

Here are the predictions for Round 19. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 2

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 7 matches played, 4 of which were correctly predicted, a success rate of 57.1%.
Here are the predictions for last week’s games.

Predictions for Round 2

Here are the predictions for Round 2. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

There’s a Herald story, from the NYT headlined Covid 19 coronavirus: Vaccine-makers keep safety details quiet, alarming scientists.

The headline  is a bit misleading. The lead says

Researchers say drug companies need to be more open about how vaccine trials are run to reassure people who are skittish about getting a coronavirus vaccine.

which is much more accurate. In fact, the information being sought isn’t mostly about safety but about how effectiveness will be judged

Another front-runner in the vaccine race, Pfizer, made a similarly terse announcement Saturday: The company is proposing to expand its clinical trial to include thousands more participants, but it gave few other details about its plan, including how it would determine the effectiveness of the vaccine in its larger study.

I wrote a bit about early stopping of trials last week. One of the main pieces of information being sought is just how the various trials will handle an unexpectedly good vaccine: how dramatic will the effect of the vaccine have to be to stop the trial early?  Normally, this wouldn’t be public information; there isn’t any compelling reason why it has to  be, but there isn’t any very good reason why not.  This case is different: the political situation has raised a real possibility that  the President of the USA would try to make the FDA authorise a vaccine without convincing evidence that it works. I don’t use words like ‘disaster’ lightly, but this would be a  disaster, both for COVID prevention and for the long-term credibility of vaccination.  We need a vaccine that works, and we need people to know that it works.

Fortunately, the FDA and the pharmaceutical companies are a bit more long-sighted (Pharma, considered as multinational public-stock corporations, is greedy, but they’re not stupid).  If the current administration tries to push a vaccine through without displaying good evidence, there will be resistance from scientists inside and outside the FDA. There are even encouraging signs that the drug companies wouldn’t ask for authorisation without evidence that at least looks moderately convincing from a safe distance.

Even so, it would be good to know the details in advance. We have the FDA Guidance document, which goes some way to nailing the FDA’s colours to the mast. There’s some public information about an NIH working group that seems to have made sensible design recommendations. But in this particular setting it would be helpful to release the details of how the trials will define success — measurements, diagnosis definitions, guidelines for early stopping — so that dozens of scientists with no connection to the trials could say, hand on heart, “that’s not exactly how I would have done it, but it’s 100% a reasonable way to define a successful vaccine”.

Update: in case it isn’t clear, I’m not saying it’s impossible for President Trump to push an ineffective vaccine through the FDA. I’m saying it’s impossible for him to do it without it being obvious to researchers in the US and around the world who he doesn’t control. And that if he doesn’t and the vaccine really works, it’s important for you and everyone else to know that.

Winston Peters, disagreeing with the continued level 2 restrictions (via the Herald)

“Travelling around the South Island has reinforced that people are not observing social distancing in the absence of any registered or real threat of Covid-19 exposure since late April.

“Not because they are against the Government’s Covid-19 response, but because they have applied their own ‘common sense’ test to their risk of exposure to the virus.”

He’s probably right. But that’s the problem.

Our own commonsense understanding of risk is pretty good.  If you’re deciding how fast it’s safe to take a winding road in bad weather, you can do well with commonsense perceptions  (unless you’re drunk or 16, and even  then you’ll probably make it).  If you’re deciding about your personal COVID risk you don’t have as much specific experience to fall back on, but there has been advice around for months and you’ve been watching the daily case counts. So, again, probably yes. The risk is low tomorrow.

But that’s not the question.  We don’t have restrictions because the risk is high. We have them because any COVID transmission will make the risk increase, slowly at first, then faster and faster over time. Since there’s a lag of a week or two in seeing the consequences, things would look the same whether we have a  long  trail dribbling off into nothing, or  a second wave all across the country. I can’t tell the difference. You can’t tell the difference.  Even Siouxsie or Ashley can’t tell the  difference!  In one case, you all in the South Island  have another moderately restricted week; in the other,  you and we go back to lockdown at huge expense and inconvenience, and potentially a bunch of people die or become chronically ill.

Because  the risk is not an immediate and visible and individual one, but a delayed and invisible and community one, commonsense and experience just doesn’t work on its own.   I don’t know whether the decision to extend level 2 was correct or not, because I don’t have all the information and modelling. Mr Peters has seen months of the best scientific and economic advice the country has to offer. He may know. But he’s not arguing based on his special knowledge  but on general commonsense risk perception. That won’t tell you.

Team Ratings for Round 20

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 114 matches played, 76 of which were correctly predicted, a success rate of 66.7%.
Here are the predictions for last week’s games.

Predictions for Round 20

Here are the predictions for Round 20. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

One of the COVID vaccine trials sponsored by AstraZeneca has been ‘put on hold’ because of  a ‘suspected serious adverse reaction’.  What does that mean and how bad is it?

Some terminology

• Adverse Event (AE): a bad thing that happens to someone in a trial. These are not necessarily related at all to the trial treatment; something counts as an AE even if you’re getting placebo. They range from minor to serious
• Serious Adverse Event (SAE): the adverse event is hospital bad.  More precisely,  fatal, life-threatening, needing hospitalisation, or leading to significant permanent effect, or  could have gotten that way if it wasn’t stopped by treatment.
• Adverse Reaction (AR): an adverse event that was caused by the trial treatment. Some of the COVID vaccine candidates have quite a  lot of these; fever, for example.
• Serious Adverse Reaction: an SAE that was caused by the trial treatment
• Suspected Serious Adverse Reaction: an SAE that might have been caused by the trial treatment
• Suspected Unexpected Serious Adverse Reaction (SUSAR): an SAE that might have been caused by the trial treatment, and wasn’t disclosed to the participant as a reasonably expected result. (Some treatments do have expected serious adverse reactions — some cancer chemotherapy, for example)

It’s standard in clinical trials that you consider SSARs/SUSARs urgently, and for completely new treatments the consideration should be especially careful.  It might turn out on investigation that the event is unlikely to be caused by the treatment (eg, because of timing, or because you find a more likely cause); it might turn out that it is likely to be caused by treatment;  or everything might just be messily inconclusive.

If you have tens of thousands of people in vaccine trials then adverse events will occur, and some of the adverse events will be serious. That’s inevitable. If you have a treatment that could potentially cause a very wide range of adverse events, then some of the serious adverse events will inevitably look like they could have been caused by the treatment.  And, sometimes, they will have been.  Vaccines aren’t intrinsically safe; they are safe when approved because they get thoroughly investigated first.

The immediate question for the trial sponsor and investigators in this case is whether the adverse event, whatever it is, significantly changes the risk:benefit balance of the vaccine candidate, as explained to the study participants (and to the company’s investors).  So far, no-one knows, but AstraZeneca seem to be handling this appropriately, and there’s no obvious reason to expect that to change.

Team Ratings for the Qualifying Final

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 20 matches played, 17 of which were correctly predicted, a success rate of 85%.
Here are the predictions for last week’s games.

Predictions for the Qualifying Final

Here are the predictions for the Qualifying Final. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 19

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 108 matches played, 70 of which were correctly predicted, a success rate of 64.8%.
Here are the predictions for last week’s games.

Predictions for Round 19

Here are the predictions for Round 19. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for the Final

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 103 matches played, 79 of which were correctly predicted, a success rate of 76.7%.
Here are the predictions for last week’s games.

Predictions for the Final

Here are the predictions for the Final. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.