March 22, 2012

Independence day

In one of our Stat of the Week nominations, the commenter takes issue with ” There is an incident [of intimate partner violence] reported to the police every 7 minutes and only 18% of cases are reported”

According to this there are 417,000 instances of physical intimate partner violence every year. With a population of around 4 million, and half women, this means that up to 40% (bad stat alert!) could be affected every year.

Leaving for the moment whether the 18% reporting rate is well-founded,  the ‘up to’ is critical here.  If the occurrences were independent, 417,000 instances would be 40%of NZ women, but of course they aren’t independent.  People don’t just abuse their partners once, they keep doing it until they are stopped.  With 417,000 instances, the actual number of women attacked per year could as easily be 4% as 40%.

A similar issue came up with the elimination of TV7, when Jonathan Coleman divided the number of monthly viewers of TV7 by four and called it an estimate of weekly viewers.  That would be appropriate if watching TV7 one week was independent of watching it the next week, but of course it’s not.  The real number of weekly viewers will be very close to 100% of the number of monthly viewers, not 25%.

The issue is very important in legal settings.  British paediatrician Roy Meadow famously wrote (and testified in court) “one sudden infant death is a tragedy, two is suspicious and three is murder, until proved otherwise”.  If sudden infant deaths were independent, then three in one family would be very rare, but we don’t have good evidence that they are independent.  On top of this,  multiple murder of infants, especially without leaving any signs, is also rare: knowing that a rare thing happened doesn’t tell you which one it was.

What provoked this post, though, was a reader’s email reminding us of the joke about the man who always carried a bomb when he flew, on the grounds that two bombs on the same airline would be incredibly unlikely.  The point of the joke is that his bomb obviously seems independent of a hypothetical other bomb, so the safety is unaffected.  In fact, if you manage to carry a bomb onto a plane it’s either a small regional NZ flight that doesn’t have screening, or there’s been a failure of the security system.  While it’s not terribly worrying that the security system failed, it does make you wonder what else isn’t being done according to plan. Perhaps you should reconsider your flight plans.


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »


  • avatar
    Thomas Lumley

    PS: You may notice that I didn’t credit the reader who emailed in by name. In general, I won’t publish the names of people I don’t know unless they explicitly say it’s ok.

    6 years ago

  • avatar

    One wonders if this is where the quotes telling us that 1 in 3 (not neccesarily the quote, I can’t recall the actual number but iirc, it always sounds quite high) New Zealanders will suffer from mental illness at some point in their life.

    6 years ago

    • avatar

      The number is 1 in 5 in the past year, about 50% over their lifetime (see here). It’s taken from a Ministry of Health report from 2006, Te Rau Hinengaro: The New Zealand Mental Health Survey.

      If it sounds high, it may be more how we define mental illness these days.

      6 years ago