July 31, 2012

Commuter survey ‘can’t be trusted’ – statistician

This just in from National Business Review – thanks to journo Caleb Allison and  NBR Online for giving us permission to upload the content, which sits behind a pay wall.

Commuter survey ‘can’t be trusted’ – statistician 

A statistician questions the validity of a survey promoting flexible working conditions for employees.

A recent survey by Regus – which describes itself as “the world’s largest provider of flexible workspaces” – said 67% of New Zealand employees would spend more time with family if they had a shorter commute as a result of flexible working conditions.

While the survey claimed to have polled more than 16,000 people in 80 countries, NBR ONLINE can reveal the company polled just 54 people in New Zealand.

Auckland University’s Dr Andrew Balemi says while it is a very low number, that alone does not suggest the poll is dodgy.

“Most people obsess about the sample size, but what I obsess about is the sample quality,” Dr Balemi says.

The only way to know if the information is credible is to know how the company undertook the survey.

However, the methodology was not included with the poll.

Dr Balemi says not only does this survey have a small sample size, it doesn’t tell the reader how it was obtained.

“In the absence of any explanation of how they’ve collected the data I wouldn’t trust this information.

“If they can’t even do that, I wouldn’t dignify it with any more consideration.”

He says the company may have a valid methodology and the poll could be worthy, but they should have included that information in the survey.

This follows another recent example of dodgy polling by the Auckland Council.

A press release claiming 63% of Aucklanders favour mayor Len Brown’s city rail loop turned out to have surveyed only 112 people.

avatar

Atakohu Middleton is an Auckland journalist with a keen interest in the way the media uses/abuses data. She happens to be married to a statistician. See all posts by Atakohu Middleton »

Comments

  • avatar

    The last sentence ignores what Andrew said about sample quality being more important than sample size.

    The Auckland Council poll appeared to have good methodology and the result tallied with other polls on the matter. In fact, NBR wrote a blog about that poll and were informed in comments that they had the story wrong but clearly they have their views firmly set.

    12 years ago