December 19, 2015

Punk’d

Earlier this year a current affairs program announced that they would have an interview with the man who didn’t get swallowed by a giant anaconda. Taken literally, this doesn’t restrict the options much.  There’s getting on for three billion men who haven’t been swallowed by giant anacondas; you probably know several yourself.  On the other hand, everyone knew which guy they meant.

There’s a branch of linguistics, called ‘pragmatics’, that studies how everyone knows what you mean in cases like this. The “Cooperative Principle” and Grice’s Maxims look at the assumption that everyone’s trying to move the conversation along and isn’t deliberately trolling.

One of the US opinion polling companies, Public Policy Polling, seems to make a habit of trolling its respondents.  This time, they asked whether people were in favour of bombing Agrabah.  30% of Republican supporters were. So were 19% of Democratic supporters, though for some reason this has been less widely reported. As you know, of course, since you are extremely well-read, Agrabah is not a town or region in Syria, nor is it held by Da’esh. It is, in fact, the fictional location of Disney’s Aladdin movie, starring among others the late, great Robin Williams.

I’m pretty sure that less than 30% even of Republican voters really support bombing a fictional country. In fact, I’d guess it’s probably less than 5%. But think about how the question was asked.  You’re a stereotypical Republican voter dragged away from quiet dinner with your stereotypical spouse and 2.3 stereotypical kids by this nice, earnest person on the phone who wants your opinion about important national issues.  You know there’s been argument about whether to bomb this place in the Middle East. You can’t remember if the name matches, but obviously if they’re asking a serious question that must be the place they mean. And it seemed like a good idea when it was explained on the news. Even the British are doing it. So you say “Support”.

The 30% (or 19%) doesn’t mean Republicans (or Democrats) want to bomb Aladdin. It doesn’t even mean they want to bomb arbitrary places they’ve never heard of. It means they were asked a question carefully phrased to sound as if it was about a genuine geopolitical controversy and they answered it that way.

When Ali G does this sort of thing to political figures, it’s comedy. When Borat does it to unsuspecting Americans it’s a bit dubious. When it’s mixed in with serious opinion polling, it risks further damaging what’s already a very limited channel for gauging popular opinion.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »

Comments

  • avatar
    simon Cameron

    I think your analysis is wrong. The issue is not that the place was from Aladdin, but that people ARE willing to condone the bombing of a place they’d never heard of. That is alarming….regardless of context.

    1 year ago

    • avatar
      Thomas Lumley

      You seem to be assuming they knew it was a place they’d never heard of, rather than believing it was one of the places where there was actual discussion of bombing.

      As I’ve written elsewhere, I think it’s the “Don’t Know”s who we really should be unhappy with.

      1 year ago

      • avatar
        simon Cameron

        Hmm, so an middle eastern sounding place name is sufficient?? I don’t think so.

        1 year ago

        • avatar
          Thomas Lumley

          “Sufficient” for what? I assuming you’re not arguing people really wanted to bomb a fictional location, so they were obviously fooled. And I can’t imagine that about a quarter of American genuinely support bombing randomly selected places in the Middle East. Presumably people thought the question meant something, but I can’t see how anyone can be sure of what possible interpretations in what proportions.

          1 year ago

  • avatar
    Stefan Peterson

    If this had been part of a research project, with suitable controls, I guess this would have been ok?

    1 year ago

    • avatar
      Thomas Lumley

      I’m not sure what would count as a control for the Agrabah question. In a sense it looks as though it was already intended as a control, the way a fictional drug might be used as a control in a question about drug use, but that’s not how it was reported.

      1 year ago

      • avatar
        simon Cameron

        A reasonable control could have been to ask an equivalent number of participants whether they think it would be appropriate to bomb ‘x’ (insert western sounding, fictitious place name here). This would have been interesting…

        1 year ago

        • avatar
          Thomas Lumley

          I would have thought you’d be trying to distinguish the anti-Muslim/anti-Arab support for bombing from the concrete question of ISIS. A western-sounding name wouldn’t help there.

          1 year ago

  • avatar
    Corey Andreasen

    The people who answered yes, who are now being made fun of, and people who sympathize with them, will be less likely to respond to surveys now. I agree with you, Mr. Lumley. This sort of behavior from supposed professionals damages us.

    1 year ago

  • avatar
    David Hood

    Well I can see the problem for survey takers, from the outside I think it provides some context on “reality basis” of poll results and how connected that is to what people are answering.

    I don’t think people will be much more embarrassed by the media coverage of this than in 2012 when PPP asked if people thought Obama was a Christian or a Muslim.

    Though their “Have you ever taken your shirt off and twisted it around your head just like a helicopter” was less contentious (15% Yes, 77% No, 8% Don’t Remember)

    1 year ago

    • avatar
      Thomas Lumley

      The problem with it as a serious question is that we’ve got know way of knowing what proportion of respondents interpreted it in the various possible ways.

      No-one would have misinterpreted the question about Obama’s religion — they might or might not have answered honestly, but they weren’t being tricked about what the question was.

      1 year ago

  • avatar
    David Hood

    Semi relevant to this discussion, the Atlantic had an article on Trump support and the poll mode (the need to tell a person you support Trump) effect http://www.theatlantic.com/politics/archive/2015/12/the-embarrassment-of-supporting-donald-trump/421365

    1 year ago