August 23, 2012

How was the survey done, by whom, and what are they selling?

Those are the basic questions you need to ask when reading, for example, that “almost half of Australian workers would rather quit a job than deal with office tension.”

Unfortunately, in the case the media release doesn’t answer the first question, and its answer to the second and third questions is

The 2012 R U OK? Australian Workplace Relationships Survey was developed in partnership with the Centre for Corporate Health, one of Australia’s leading workplace mental health service providers. 

Now, workplace mental health is an important issue, and for all I know it may be seriously neglected in Australia (I last worked there in 1995, but it was in a medical research centre, which probably isn’t typical).   That doesn’t mean mental-health advocates get a free pass on data quality — good facts matter more for important issues than for, say, whether we should wear pyjamas in the street.

The Centre for Corporate Health isn’t that enthusiastic about telling us how the survey was done: you need to subscribe to their research news to get in, and even then all they say is

The survey was distributed via online and paper-based to people across a wide range of Australian workplaces including both public and private sectors with all industry groups being represented.  The survey was open for a four week period. The total number of survey respondents was 1554.

which is not encouraging either in what it says or what it omits.  As far as I can tell, there’s no reason to believe the numbers are good estimates of anything about Australian employees as a whole.

The reason that organizations do these surveys seems to largely be marketing: having things that look (from the right angle, in poor light) like statistics makes it easier to get media coverage.  It shouldn’t: if we’re covering West Island business or health news, “R U OK Day” is important enough to be in the papers without needing unconvincing numerical camouflage to get it past the gate.

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »