July 26, 2012

Good reporting of a poll

The Herald has a fairly good story about a poll on motorway tolls:  the target population (Auckland City), the sample size, the results, and the sampling method are all described.

The sample wasn’t a random sample, but according to the description, it was still a reasonable sample

Those surveyed were drawn from a research panel recruited by Horizon in accordance with the demographics of Auckland’s adult population at the last Census, weighted to match age, gender, personal income, employment and education levels

The respondents will be a biased sample, but the pollers have made efforts to correct this bias.  That’s what is done in good-quality national surveys even if a random sample is taken, because non-response inevitably makes the sample less representative than it should be.

There are two points that would have been welcome in the story which weren’t there.  Firstly, there is no margin of error.  Using a biased sample and reweighting will typically give a larger margin of error than using a random sample, though not so large as to make a big difference to the conclusions the Herald reports.  Secondly, the respondents were asked for their opinions about different ways of raising 10 billion dollars to pay for major city projects, but it’s not clear whether they were asked about just not doing the projects and saving the 10 billion dollars.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »