Stat of the Week Winner: May 12-18 2012

Thanks for all those who sent in a nomination last week for our Stat of the Week competition. Please enter this week’s competition too!

This week’s winner has been chosen to be Nick Iversen’s nomination of the New Zealand Herald’s editorial which says “Self-selecting polls show which option has the most supporters who care enough to cast a vote”:

The Herald appears to be trying to justify why they can headline bogus polls as news (https://www.statschat.org.nz/2012/05/13/bogus-polls-treated-as-news/).

But a self selecting poll does not in fact show which option has the most supporters among those who care to cast a vote.

The self-selectors did not select themselves based on their level of care. They selected themselves based on having internet access, reading the Herald web site, and willingness to participate in a bogus poll. They don’t represent any meaningful segment of NZ.

The poll is still bogus (even if they published the sample size).

James Russell from our department explains why this is our winner this week (even though there were some fantastic other nominations!):

Self-selection is one of many biases that can affect the results of a survey. In this case the winning post highlights that self-selection can have many filters imposed on it, including motivation to respond to the survey as a final probability filter (e.g. x% of people), however prior to that there are other filters such as access to the internet (z%) and reading the Herald website (y%).

In this simple case the probability of a random person responding is already at least a function of z% x y% x x%. This is certainly not a random subset and is unlikely to be a meaningful one of society. Internet polls can struggle from other fascinating biases such as viral hacking. These biases can never be completely avoided in any survey but any statistician should have some idea of the level of them and their potential impact on the results.

As the winning post, this highlights a methodological issue underpinning many of the statistics discussed on Stats Chat.

Congratulations Nick Iversen!