May 2, 2012

Survey respondents say the darndest things

Stuff is reporting a mind-boggling survey result

Nearly 15 per cent of people worldwide believe the world will end during their lifetime and 10 per cent think the Mayan calendar could signify it will happen in 2012, according to a new poll.

The obvious expectation is that this is a Bogus Poll and that nothing of the sort is true.  However, as the story says, this is a poll conducted for Reuters  by Ipsos, and the findings as given by Ipsos are just as Stuff reports them.

Presumably Ipsos, who know how to poll, are accurately reporting what people said, and the 15% and 10% figures are actually reasonably representative of what people will answer if you ask them that question.  That doesn’t mean the conclusions are true — they obviously aren’t true, since if 15% of people really believed that, they would be behaving differently.

This is a big problem with surveys — even representative polls of people can give weird results, because polls measure how people answer questions, not what they actually believe.

One of my favorite examples is a poll (via) conducted shortly after the massive  Gulf of Mexico oil spill, which found that  “28 percent of Republicans said the recent oil spill in the Gulf of Mexico made them more likely to support drilling off the coast“. This just makes no sense: you could rationally believe that the oil spill is just part of the costs of economic growth and that it doesn’t change your opinion, but supporting drilling more, because of a drilling accident that turned out to be worse and harder to fix than expected, is insane.

 

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »

Comments

  • avatar
    Rachel Cunliffe

    Interesting that the first belief is a “will” and the second is a “could”.

    The 10% is not really too shocking then, as you’re not committing to anything.

    For the 15% who believe the world will end during their lifetime, is that much different from believing that you are going to die in the next 100 or so years? I.e. would it really change your behavior that much?

    Also, perhaps that sort of belief *has* changed their behavior already, we don’t know?

    (Great post though, has got me thinking!)

    12 years ago

  • avatar
    Steve Taylor

    I’m curious to know how to get a representative sample of “people worldwide” i.e. so that every living person has a positive probability of being sampled?

    12 years ago

    • avatar
      Thomas Lumley

      What they actually mean is a representative sample from a collection of countries. It’s “worldwide” in the same sense as “20% off storewide”

      12 years ago

  • avatar

    I don’t believe that the world is going to end, at least not until a supernova hits well outside of any reasonable time horizon.

    But there are non-insane ways of holding that belief. Here’s one.

    The great filter. The probability of life starting on any planet is small, but there are lots of planets that are in the potentially habitable range. And yet we haven’t observed any other life. Something has stopped older planets from establishing observable galactic civilizations. That filter could be behind us – maybe the move from bacteria to multicellular life is harder than we think (then again, you can get from yeast to multicellular cooperative structures surprisingly quickly) – or it could be ahead of us. The closer we get to feasible interstellar travel, the closer we’re getting to whatever stopped anybody else from getting there first. See Hanson, here: http://hanson.gmu.edu/greatfilter.html

    I doubt that 1/10000 potential respondents were thinking something almost-sane like the Great Filter though.

    12 years ago