Posts written by Thomas Lumley (2645)

The Herald has a headline “Wanna live longer? Look on the bright side”.  The story goes on to say

A study of 243 centenarians revealed that most were cheerful and sociable. The positive personality traits may be genetically based, the researchers believe.

If it’s really genetics that produces the positive personality traits, the advice isn’t going to do much good.  The research paper that produced the story is here, for comparison

The story also says

The study involved Ashkenazi Jews from eastern Europe.

Actually, from the US, though many of them were migrants from central and eastern Europe.  Then

They are good candidates for gene studies because they’re genetically similar to one another.

which is true.  But this wasn’t a gene study, and the relative genetic and environmental homogeneity of the Ashkenazim make them a particularly bad choice for a study that compares Ashkenazi centenarians with the US population of all ages.

In fact, the point of the research paper wasn’t to compare centenarians with others.  The main point of the research paper was really to describe the questionnaire that the researchers are using in their genetic study and compare it to other questionnaires measuring similar psychological traits.  We can see this from the concluding sentence in their abstract:

Findings suggest that the POPS is a psychometrically sound measure of personality in centenarians and capture personality aspects of extraversion, neuroticism, and conscientiousness, as well as dispositional optimism which may contribute to successful aging.  

That is, the researchers assume that these personality aspects will turn out to be important (in the genetic studies) and are looking at how well they are measuring these traits.  That’s all perfectly reasonable, it’s just strange to see how it morphs into the newspaper story.

As you will have heard, excises on smoking are going up.  This will raise money (to the extent that smokers don’t quit) and reduce smoking (to the extent that they do quit).   If you’re interested in the modelling used to estimate the impact on these conflicting goals, the Treasury’s Regulatory Impact Statement is a well-written and detailed explanation.

It’s also interesting to note that Treasury agrees the excise costs are already probably higher than the costs to other people imposed by smoking, and since the smoking excise is probably a regressive tax, the only convincing motivation for smoking excise taxes is to stop people smoking and so improve population health in the long term.

In this light it’s interesting that the Herald’s bogus poll for today is on whether increasing costs will lead to fewer smokers: at the moment, only 11% of responders think it will.  Fortunately, there is strong evidence that the poll respondents are wrong.

A blog post by a veteran editor, on one of the recurring stories about ‘lucky’ lottery-ticket outlets, explains elegantly and simply why it’s a real problem:

In some cosmic sense, sharing popular delusions about the lottery isn’t any worse than publishing the horoscopes. But if you’d like to be taken seriously when you proclaim “study says” or “poll reveals,” you need to run a disclaimer with your “want to hit the lottery” tales:

“We’re spending $70 per person to fill this out. That’s just not cost effective,” he continued, “especially since in the end this is not a scientific survey. It’s a random survey.”

That’s US Congressman Daniel Webster, quoted in the New York Times talking about the American Community Survey, the more-detailed part of the US Census that is administered to a 2% sample of Americans each year.  As you know, the fact that the ACS is based on a random sample is what makes it a scientific survey, and it is a very valuable one to US government and business as well as to researchers.

If the NZ census goes to a ten-year rather than five-year cycle, an interim sample similar to the ACS would be one way of maintaining up-to-date and accurate information at reasonable cost.  Or we could just leave information collection to Facebook and Wikipedia.

There were two Stat-of-the-Week nominations (from Patricia de Guzman and Samantha Post) for the Conservative Party’s use of the Durex Sex Survey to oppose providing subsidised contraception.  We’ve seen this survey number before on StatsChat and it was in the NZ media when the figures were released, back in 2007.

What’s really striking about many of  the stories is the focus specifically on promiscuity of women, when the number that the survey was claiming to estimate is the same for heterosexual men and heterosexual women as a simple matter of arithmetic. 

As is so often the case, the political position seems to be developed independently and statistics sought to drape around it.  Or as Andrew Lang put it  they “use statistics as a drunken man uses lamp-posts — for support rather than for illumination.”

In the Stat of the Week competition, Nick Iversen points to a poll with a margin of error given as 3.9% from a sample of 1004, where the simple formula based on sample size would give 3.1%, and writes

The US poll uses 3.9%. I wonder if they are accounting for sampling bias as well as sampling error. Sounds to me like the US pollsters are doing a more sophisticated measure of error than NZ ones.

Following the link to the pollers methodology, we find that weighting is responsible.  The poll answers are reweighted to known population totals (eg, from the census) for age, sex,  and race/ethnicity, and also reweighted based on phone numbers per household.  They say

“Once weights are final, the effective, or weighted, sample size – rather than the actual
sample size – is used to compute the survey’s margin of sampling error.”

So, the 3.9% maximum margin of error doesn’t account for sampling bias in any direct way (which would be hard to impossible), but it corrects for the reweighting that is done to reduce sampling bias.  The more variable the resulting weights, the lower the effective sample size and the higher the maximum margin of error.

There’s some discussion on Simon Jackman’s blog (Australian political scientist now at Stanford) saying that serious US polling organisations usually do adjust the margin of error because of the reweighting, but that Australian ones don’t seem to. It looks like NZ ones don’t either.

In the past, newspapers would usually at least refer to their bogus polls of convenience samples as “unscientific”.  Now the warning word simply seems to be “online”.  The Herald’s headline “Mother’s Day off the cards for most: poll” is followed by the lead “Almost half of New Zealanders won’t be celebrating Mother’s Day today, according to a New Zealand Herald online reader poll”.

If the poll were meaningful, it would also matter that the question is badly designed: there are four answers, of which you have to pick exactly one to vote.  The options are a card, a meal, breakfast in bed, or nothing.  If you’re taking Mum skydiving, or sending flowers, or buying her chocolate, or a new chainsaw, how are you supposed to respond?

With the PBRF research assessments happening this year, there is sure to be another round next year of universities using the results in creative ways to make themselves look good.  When you have a large number of variables to take into account, it’s easy to come up with apparently-reasonable weightings that make one institution look better than another.

A dramatic example of this is ratings of US law schools.  The most widely-known rankings are from US News, and there’s another popular set from Brian Leiter, a professor at University of Texas, which aims to focus only on quality of education.  A third set is published by the slightly unusual Thomas J. Cooley School of Law.    There is definitely correlation in the top rankings: for example, Harvard tops the Cooley rankings and is second for the other two.   The graph below (click to embiggen) shows the three rankings and the lower quartile of GPA and LSAT results for some top colleges (I got the data from here, and added the Cooley rankings by hand)

You were wondering perhaps about the two colored dots?

The orange dot is University of Texas, Prof. Leiter’s employer.  His ranking for his own school is between the other two rankings.  The red dot is Thomas J. Cooley law school.  The US News rankings don’t  include them in the top 100 (they don’t quantify past the first 100), but their own ranking makes them second in the US, behind only Harvard.

The Herald has a story claiming that a set of chemicals that have been proposed as an antibacterial additive for meat actually have large effects on sex-hormone levels.  Usually this would be a story about the need to ban the chemicals immediately, but this time the headline is “‘Viagra effect’ from pomegranate juice”, and they’re in favour of it.

Of course, the ‘Viagra effect’ label is completely bogus (the quote marks suggest that it comes from the researchers’ press release, which would be very dodgy if true).  The researchers claim to have found an increase in testosterone levels in men and women who drank a daily glass of pomegranate juice.  Viagra is involved in blood vessel dilation; it has nothing to do with testosterone, and a previous suggestion that pomegranate juice might really have Viagra-like effects has been tested and rejected.

So, what about the testosterone effects?  Well, what we have is a story based on a press release about a small, unpublished, uncontrolled, open-label study. The most positive one could possibly be about this is “It will be worth waiting for the real publication” or,  perhaps, “I hope it’s not true, because messing with steroid hormones like that is scary”.  It’s reassuring to note that last year the same group said pomegranate juice reduced office stress.  In 2009, they said it reduced blood pressure and cortisol levels.   You will notice that the last link is a press release from a manufacture of pomegranate juice, who sponsored all these studies.  I haven’t linked to actual publications, because according to the PubMed database neither of these results has yet been published either.

There’s a lot of this around: a different pomegranate manufacturer has been sued by the US Federal Trade Commission (in 2010).  The Wall Street Journal wrote

The FTC’s target is not POM’s generally worded, eye-catching ads with lines such as “Cheat Death,” said an agency official. That ad showed a bottle of pomegranate juice with a rope around its neck.

Instead, the government complaint is directed at more specific health claims.In addition to talking about arterial plaque and blood flow, some POM ads describe company-funded clinical trials that, according to the company, show POM juice products can slow the progression of prostate cancer by lowering the level of antigens in the body called PSAs.

The agency’s complaint says that some of POM’s studies did not show heart disease benefits and that the prostate cancer study wasn’t conducted in a standard, scientifically rigorous manner.

The findings about pomegranate juice could be true, but it’s clear that the target isn’t people who actually care whether they are true.

Stuff has a story on a new genetic finding:  the cause of (naturally) blond hair in Melanesians, which turns out to be different from the cause in Europeans. You can read the full paper at Science (annoying free registration)

The researchers looked at DNA samples from 43 blond and 42 dark-haired Solomon Islanders. First, they looked at a grid of DNA markers that are relatively easy and cheap to measure.  This pointed out a region of the genome that differed between the blond and dark-haired groups.  The region contained just one gene, so they were then able to determine the complete genetic sequence of the gene in 12 people from each group.  This led to a single genetic change that was a plausible candidate for causing the difference, which the team then measured in all 85 study participants, confirming that it did determine hair color.

The result is interesting, but not all that useful. The gene responsible was already known to be involved in hair and skin color, in humans and in other animals, so that didn’t provide anything new, and if you want to find out or change someone’s hair colour, there are easier ways.  Still, it’s interesting that blond hair has appeared (at least) twice — most traits with multiple independent origins are more obviously useful, like being able to digest milk as an adult, or being immune to malaria.

This general approach of a wide scan of markers and then follow-up sequencing is being used by other groups, including a consortium I’m involved in.  Although it worked well for the hair-color gene, it’s a bit more difficult to get clear results for less dramatic differences in things like blood pressure. Fortunately, we have larger sample sizes to work with, so there’s some chance.   We’re hoping that finding the specific genetic mechanisms behind some of the differences will provide leverage for opening up more understanding of how heart disease and aging work.  If we’re lucky.