Posts written by Thomas Lumley (2645)

avatar

Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient

June 7, 2012

Qualitative vs quantitative

The Herald claims “NZ says no to larger schoolrooms”  based on a street survey of “more than 70” people, of whom 81% were opposed to the changes.   The current clicky poll has 74% of about 8000 responses supporting the much weaker claim ‘Less one-on-one time can’t be good for kids’,  a statement that even John Key would probably not contest. We aren’t told what the actual questions were in the street survey, or how much the respondents knew about what the actual proposed changes were.

There are two ways you can get useful data by interviewing people.  In quantitative research, where you take a proper probability sample and ask questions with simple answers unambiguously related to what you are trying to find out.  The law of large numbers then ensures that your sample results are not too different from the population results.   The Colmar Brunton polls are an example of this. In qualitative research, you are trying to find out the full range of responses and understand people’s thinking: you get smaller numbers of people and ask much more open-ended questions.  Marketing focus groups are an example of this approach.

Just as the clicky website polls are a degraded version of quantitative survey research, the street poll is a degraded form of qualitative survey research.  A good qualitative survey would try to find out how people feel about the tradeoffs in education funding, about where they would rather make cuts, and how views differed between groups of people — do people without children give similar explanations to  those with children, for example.

In this case, however, public opinion is probably clear enough that any measurement will give a similar result, at least as long as you’re not interested in working out what policies would be better.

June 6, 2012

Equal pay statistics

As we know, women are paid less than men, and it’s not primarily because of sick-leave differences.   So what does cause it?  Motivated by current US legislation, the Washington Post digs out a 2007 research report that estimates how much of the difference can be statistically explained by factors such as choice of occupation and leaving the workforce to take care of children.

In a sense, this attempts to give a lower bound on the impact of discrimination —  some of the impact of childcare responsibilities is a hangover from traditional roles, but it’s quite possible that in a perfect world there would still be some gender differences in child care.  Similarly, some of the lower pay in majority-female occupations is probably because these occupations had more women, but it’s hard to say how much of it.

The reseachers, Francine Blau and Lawrence Kahn, looked at US data from 1979, 1989, and 1998 (the report was being prepared and revised for a long time). In 1979 the pay ratio was 63%, but comparing men and women with the same education and experience it was 71%, and additionally controlling for occupation, industry, and union coverage it was 82%.  In 1998 the numbers were 80%, 81%, and 91%.

So, by 1998 about 10% of the US pay difference between men and women was explainable by differences in education and experience, about half of it was explainable by working in different industries or occupations, and the rest was not explainable by anything they measured.    I don’t know if anyone has done a similar analysis for New Zealand, where the differential is quite a bit smaller than in the US.

This is an example of the sort of thing you can only do with good-quality survey data; in this case from the University of Michigan’s Panel Study of Income Dynamics.

June 5, 2012

Think of a number, then multiply by five

You will have seen the controversy about the number of jobs in the new Sky City convention center.  I don’t have anything to add on the number of employees after it is built, but it’s interesting to see how construction jobs seem to be defined

The company’s report says 150 construction jobs could be created each year over a five-year period, making a total of 750, but they would be filled by people already employed on other projects. (3 News)

The original Horwath report said 150 jobs could be created over a five-year construction period for a total of 750.(Herald)

That is, if you have a five-year construction project that at a typical moment employs 150 people, this creates 750 jobs.  If you have a factory, or an office, or a convention center that employs 150 people at any time you would usually say it created 150 jobs — even that wouldn’t actually be true, since it mostly relocates jobs rather than creating them, but at least it’s fairly clear what is meant.

We should clearly hope that the convention center takes more than five years: the longer the construction period, the more jobs are created.

Why European Union is hard (part II)

One of the difficulties with surveys on sensitive questions is that people may respond just out of emotional affiliation or based on slogans, rather than actually reflecting carefully on their beliefs.   That’s the positive interpretation of opinion poll results collected in February by a Greek public opinion firm, who appear to be respectable apart from their horrible taste in graphs (via, and)

Among the questions was a section where respondents were asked whether they Strongly Disagree, Disagree, Agree, or Strongly Agree with statements (presumably in Greek and translated in the report)

  • Greece should claim by any means from Germany war reparations/indemnities
  • Greece should claim by any means from Germany the payment of ‘war loans’ granted to the German Occupation Forces during the period 1941-1944
  • Germany, with its current policy, attempts to dominate Europe through its financial power
  • They have right all those who argue that Germany’s current policy aims at the establishment of a Fourth Reich.

It’s surprising that someone would ask these questions, but it’s even more surprising that the proportion agreeing was 91%, 87%, 81%, and 77% respectively, and mostly in the ‘Strongly Agree’ category.

Why European Union is hard (part I)

A graph originally from JP Morgan, and now around the web (via, and via) comparing the social, economic, and political heterogeneity of the Euro zone to various hypothetical monetary unions, some more plausible than others.  The Eurozone is more heterogenous than all of them.

June 3, 2012

So that’s what chocolate is for!

Q:  Have you seen the  new story in Stuff about chocolate being good for your heart?

A: Indeed.  A fine example of the genre.

Q:  What did the researchers find out about the benefits of chocolate this time?

A: Absolutely nothing. (Here’s the research paper)

Q: Then what is the story about?

A: They assumed that there were significant cardiovascular benefits of high-cacao dark chocolate, based on short-term studies of blood pressure and cholesterol, and worked out what the population impact of chocolate therapy would be if that was true.

Q: And is it true?

A: That’s not clear.  If you remember the last time we encountered ‘chocolate for health’, the benefits in that study were pretty marginal. This time the researchers cite a pooled set of studies giving a somewhat larger estimate of the benefits.   Also, the research showing benefits on blood pressure and cholesterol has been from very short-term studies, and extrapolating to the longer-term may well be unreliable.  Still, as Prof Reid points out, no-one is going to do any reliable studies, so extrapolation is what we’ve  got.

Q: What about the calories and fat?

A: Well, the studies that provided the estimates of benefit compared chocolate or cocoa powder to a control that supplied the same calories and fat, so we don’t know.   The researchers (and even the newspaper) admit “The study did not take into account weight gain … ”but that is something you would need to consider,” [Prof Chris Reid]  said.”

Q: Ok, if we stipulate all of that, how big were the benefits?

A: Moderate:  treating 10,000 high-risk people with chocolate would, under the most favorable assumptions, prevent about 15 fatal and 70 non-fatal heart attacks and strokes over a ten-year period.

Q: How do the benefits compare to pills?

A: Well, thiazide diuretics provide 2-3 times the blood pressure effect, and simvastatin provides maybe 10 times the cholesterol reduction. And they cost Pharmac less per month than the chocolate would cost you per day. And we really know the reductions in risk factors translate to fewer heart attacks, strokes, and deaths

Q: But didn’t the researchers say that chocolate therapy would be cost-effective?

A: They said it would be cost-effective at a cost of $42/year.   According to their website, if you wander down to your local Countdown, you can currently get Whitaker’s Dark Ghana at $3.79 for 250g. That would be $550/year for the 100g/day dose in the paper. And that’s a sale price.  It’s only cost-effective if you don’t count the cost of the chocolate.

Q: How about cocoa powder?

A: One of the studies that went into the analysis used 31g/day of cocoa powder (`natural’ not Dutch process). That’s cheaper, but it’s still a lot more than $42/year, and 5 tablespoons/day is a lot of unsweetened cocoa powder.

Q: So, should I eat dark chocolate?

A: If you like it, sure.  If you’re just choking it down for the cardiovascular benefits, probably not.

 

May 31, 2012

P value falls, use also falls

An interesting report in the Herald yesterday on methamphetamine use.   A forthcoming Health Ministry survey apparently shows a decrease in the proportion who report using P over the past year, from 2.1% in 2007/2008 to 1% now (yes, that will be larger than the margin of error; it’s a big survey).

On the other hand, an ongoing Massey University study that interviews police detainees about drug and alcohol use finds that the street price and availability of P have not really changed.

This evidence of a decrease on the demand side is encouraging for two reasons.  First, it suggests a decrease in the number of people who are dumb enough to want to take methamphetamine, which has to be progress.  Second, it suggests that the pseudoephedrine ban wasn’t responsible, as that should have acted on the supply side by driving up street prices.  Can we have our Sudafed back?

May 30, 2012

Tracking a press release

The Herald has a story about curcumin, a component of turmeric (which, as you know, is what makes curry stains hard to get rid of).  Apparently it also has some interesting effects on  a component of the immune system. The story appears in quite a few newspapers around the world, and in Science Daily and Medical Daily, which collect this sort of story.

Often it’s hard to tell how much the stories depend just on the press release or whether someone at one of the specialised sites has actually taken a look at the paper.  It’s not hard in this example, though: the paper hasn’t appeared yet.  There seems to be some slipup at the journal. The university press release and all the sites say the paper came out Friday 25th May, but the Journal of Nutritional Biochemistry hasn’t gotten to Friday yet: at the time of writing this they are still on Thursday 24 May.

May 29, 2012

Getting it right, again

The Herald had a  clicky poll on the proposed R-plate restricted-driving legislation, and has reported the results in an unobjectionable paragraph at the end of a good story getting input from experts.  I’m not sure that the poll actually adds to the story, but at least it doesn’t subtract from it.

May 28, 2012

Getting it right

You can report a self-selected sample responsibly: the Otago Daily Times shows how. Their lead:

Ninety percent of people who have made submissions on the Queenstown Lakes District Council representation review want to keep an Arrowtown ward and councillor.