November 5, 2022

Winston First?

An ongoing theme of StatsChat is that single political polls aren’t a great source of information, and that you need to combine them. A case in point: this piece at Stuff describing a new Horizon poll.  The headline is Winston Peters returns to kingmaker position in new political poll, and the poll has NZ First on 6.75%.  My second-favourite NZ poll aggregator, Wikipedia, shows other recent polls, where the public results from Curia, Roy Morgan, and Kantar were 2.1%, 1%, and 3% and a leaked result from Talbot Mills was 4%.  It’s possible that this shows a real and massive jump over the past couple of weeks. Stranger things do happen in politics — but not much stranger and not all that often. It’s quite likely that it’s just some sort of blip and doesn’t mean much.

Stuff does add “The poll had a margin of error of 3.2%, meaning NZ First’s crossing the 5% threshold was within the margin of error,”  but that’s the wrong caveat.   The 3.2% margin of error is more strictly called the ‘maximum margin of error’, because it’s the margin of error for proportions near 50%, which is larger than at, say, 5%.  I’ve written before about calculating the corresponding margin of error for minor parties.

In this case, under the pure mathematical sampling approximations used to get 3.2%, a 95% uncertainty interval for NZ First’s true support would go from 5.2% to 8.5%. If we only worried about sampling error, NZ First would be fairly clearly above the 5% threshold.  The problem is that the mathematical sampling error  is typically an underestimate of total survey error — and when you get a very surprising result, it’s sensible to consider that you might possibly be out on the fringes of the total survey error.  Or not. We will find out soon.

 

 

 

 

 

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »

Comments

  • avatar
    Scott Brooker

    I saw this too. My guess is that the cause is this sentence buried pretty deep in the article. “The outcome comes from a survey 911 adults who were registered to vote and, unlike other major polls, were certain they would vote.” Hypothesis for testing: Minor party supporters are more likely than Nat/Lab supporters to express certainty about voting at this early stage.

    1 year ago

  • avatar
    Steve Curtis

    Doesnt Horizon soley use online panels of voters who self register .
    Horizon uses prizes as click bait to get its panellists to complete its endless series of mostly push polling for its clients ( your soap, your power company, your pet food etc) with the political poll just done for brand awareness in the wider media
    One recent prize was ‘$1,000 cash and the latest iPad Pro (more powerful than most laptops) worth $1,199! Or you can take the lot in cash: $2,199!’
    Its self selected panel who respond to the bait are those who like prizes!

    1 year ago