December 20, 2020

Good enough that you don’t need statistics?

From the usually-reliable XKCD

The graphs of cumulative Covid incidence for the Pfizer and Moderna vaccine trials are very impressive: the vaccines are effective.  We need to relabel the axes to see what’s wrong with the cartoon

If the trials had just been about vaccine effectiveness, they would have been much shorter.  Delaying the actual use of the vaccine, when the US alone is getting 1.5 million cases a week, just in order to get a pretty graph and avoid using statistics would be … ethically questionable. No matter how you felt about your high school stats class. If we only cared about effectiveness, the graph would look more like

The reason the trials ran so long was to assess safety.  Everyone in the trials was followed up for six weeks after their second vaccine dose, because that’s long enough to see any adverse reactions that have been seen for past vaccines.  The very high rate of new cases in the US and other countries involved in the trials, together with the high effectiveness, meant that effectiveness could be demonstrated much more quickly than safety — but safety is important.

Safety is especially important for us in NZ, where there is less urgency than in the US or UK.  Medsafe will be looking at whether to approve the vaccines and under what conditions, and they’ll be doing more than “We reject the null hypothesis based on the ‘hot damn, check out this chart’ test.”

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »

Comments

  • avatar
    Joseph Delaney

    I actually think the high efficacy of the vaccine is another reason that NZ should be very focused on safety. If there are high risk but small groups who do very badly upon being vaccinated, it could be interesting to identify these populations in advance. With 95% efficacy, presuming it holds up over time, one could imagine effective herd immunity with only vaccinating those who can be safely vaccinated.

    3 years ago

    • avatar
      Thomas Lumley

      Yes, that’s another reason.

      If the new UK strain really is that much more transmissible, the herd-immunity threshold will be higher, which limits the margin for this sort of strategy. And we haven’t seen effectiveness data for any of the vaccines that will be used for the main immunisation effort here (well, preliminary for the AstraZeneca one). We’ve only got 750k courses of the mRNA vaccines on order.

      3 years ago