May 7, 2013

Not adding up

As you know, the petition for a referendum over asset sales has not reached its goal yet, due to lots of invalid signatures. This is not a new problem — the petition over the anti-smacking law initially had 17% invalid signatures and also fell short of its threshold on the first round — but it does seem to be worse than usual.

3News displayed this graph of the shortfall

petition shortfall


It seemed to me that the 16,500 bar was a bit wider that I’d expect, so I checked on the video from the website.  On my screen capture, which I think is what you get if you click on the image, the black bar has 872 signatures per pixel, the blue bar has 1018 signatures per pixel, the whole red bar has 535 signatures per pixel, and the 16500 shortfall has 232 signatures per pixel.  That is, the vertical scale for the shortfall is about four times that for the valid signatures.

I’m really not accusing 3News of deliberately distorting the numbers — it looks to me as if the shortfall bar has been made the right height to contain its text, that the blue+red bars height is scaled to the available screen estate, and that the black bar is scaled to the total blue+red height .  But it’s a pity that the result is to amplify the visual size of the shortfall — and if the visual size weren’t important the graph would be a complete waste of time.

Scaled in proportion, the bars look like this




Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »


  • avatar
    Simon Connell

    Aside from the size issue you point out, I think there’s a problem with placing the numbers/label inside the column in the TV3 graph: it might lead the reader to think that “shortfall” is a separate category of signatures, distinct from “invalid” and “valid”.
    This could be remedied by the label identifying the shortfall being placed between the two columns.

    4 years ago

  • avatar

    On the topic of the petition, I’d like to know the sampling size and methodology used.

    The press release from the Clerk says they undertook “a sampling methodology” but there’s no further information on that methodology, or sample size, so we can see how they arrived at the 16500 figure.

    4 years ago