August 15, 2012

Is that a kiwi in your pocket?

Stuff has a story based on the latest release from the “Mega Kiwi Sex Survey”, covering all sorts of headline-worthy topics such as infidelity rates, and major turn-ons and turn-offs.

Conducting an accurate and reliable sex survey is difficult: both in taking the sample and in persuading people to give honest answers.  It’s much easier not to bother with all that.  We’ve commented adversely on the Durex Sex Survey before, but that at least made real attempts to get a representative sample.  Durex hired Harris Interactive, who are probably the leaders in trying to get reliable data out of online surveys.

The Mega Kiwi survey, not so much.  The producers say (update: most links here NSFW)

Fitzgerald says they are aiming to capture data from a broad cross section of New Zealanders “We’re working with a couple of key partners to get their audiences to take part, but we’re really trying to build a complete picture of the New Zealand sexual identity, so whether you’re a 60-year-old male in Eketahuna or a 22-year-old female in Ponsonby, we want to hear from you.”

and that’s how they went about it. A little Googling finds some of the links to the online survey form.  It’s a typical bogus poll.

Even if it were a real poll, the sample size of 1500 wouldn’t justify quoting results to the nearest tenth of a percentage point, since the margin of error would be about 2.5%.    In fact, the news release gives numbers to a tenth of a percent even for Gisborne.  Gisborne has about 1% of the Kiwi population, so a representative sample would have just 15 Gisborne respondents.

Still, the real point of the survey is to get news coverage rather than having to pay for advertising.  It seems to have worked.


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »