April 23, 2013

When ‘self-selected’ isn’t bogus

Two opportunities for public comment that will expire soon, and where StatsChat readers might have something to say

  • Stats New Zealand wants to hear from people who use Census data.  They have a questionnaire on how you use the data, and how this might be affected if they change the Census in various ways. It’s open until Friday May 3
  • Public submissions on the new ‘legal highs’ bill close on Wednesday May 1.  The bill is here. You can make a submission here.  The Drug Foundation have a description and recommendations here

This sort of public comment is qualitative, rather than quantitative.  Neither the Select Committee nor Stats New Zealand is likely to count up the number of submissions taking a particular view and use this as a population estimate, because that would be silly.  What they should be aiming for is a qualitatively exhaustive sample, one that includes all the arguments for or against the bill, or all the different ways people use Census data.


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »


  • avatar
    Simon Connell

    Politicians like making something of what the majority of submitters thought, though. Here are some examples (these were just some ones that were easy to find via Google, I’m not meaning to imply that only the left does this):

    “The overwhelming majority of submissions on the latest GE bill oppose releasing GE into the environment, Green Co-Leader Jeanette Fitzsimons said today.”

    “The National Government’s decision to oppose Maori seats is in direct opposition to the majority of submissions to the Auckland Governance Legislation Select Committee, says Labour’s Associate Local Government and Associate Pacific Affairs Spokesperson Su’a William Sio…
    “Those submissions were from people and organisations from all walks of life, cultures and all sectors of our society, including some traditional National voters, and they overwhelmingly said ‘yes’ to Maori seats.”

    “Over 13,500 of around 17,000 submissions on the Employment Relations Bill support the proposed legislation, says Employment and Accident Insurance Legislation Select Committee chair Graham Kelly…
    “If we measure the public’s response to the Bill by the preferences shown in the complete range of submissions, then they are overwhelmingly in favour of the Bill,” says Mr Kelly.”

    4 years ago

    • avatar
      Thomas Lumley

      Sadly, yes. But that isn’t the point of public submissions, it’s a side-effect. And we will mock them for it when they do it.

      4 years ago

  • avatar
    Joseph Delaney

    I am also not sure that it is all bad for the people who USE census dato be over-sampled in the responses. While I know that there is no way to weight responses based on personal impact, it is often the case that public policy has differential impacts on members of the population.

    So far the default way of handing this in the US seems to be for those most impacted to sue, which seems like an inefficient way of over-weighting the responses

    4 years ago