Posts filed under Research (174)

November 25, 2015

Why we can’t trust crime analyses in New Zealand

Jarrod Gilbert has spent a lot of time hanging out with people in biker gangs.

That’s how he wrote his book, Patched, a history of gangs in New Zealand.  According to the Herald, it’s also the police’s rationale for not letting him have access to crime data. I don’t know whether it would be more charitable to the Police to accept that this is their real reason or not.

Conceivably, you might be concerned about access to these data for people with certain sorts of criminal connections. There might be ways to misuse the data, perhaps for some sort of scam on crime victims. No-one suggests that is  the sort of association with criminals that Dr Gilbert has.

It gets worse. According to Dr Gilbert, also writing in the Herald, the standard data access agreement for the data says police “retain the sole right to veto any findings from release.” Even drug companies don’t get away with those sorts of clauses nowadays.

To the extent these reports are true, we can’t entirely trust any analysis of New Zealand crime data that goes beyond what’s publicly available. There might be a lot of research that hasn’t been affected by censorship and threats to block future work, but we have no way of picking it out.

November 13, 2015

Blood pressure experiments

The two major US medical journals each published  a report this week about an experiment on healthy humans involving blood pressure.

One of these was a serious multi-year, multi-million-dollar clinical trial in over 9000 people, trying to refine the treatment of high blood pressure. The other looks like a borderline-ethical publicity stunt.  Guess which one ended up in Stuff.

In the experiment, 25 people were given an energy drink

We hypothesized that drinking a commercially available energy drink compared with a placebo drink increases blood pressure and heart rate in healthy adults at rest and in response to mental and physical stress (primary outcomes). Furthermore, we hypothesized that these hemodynamic changes are associated with sympathetic activation, which could predispose to increased cardiovascular risk (secondary outcomes).

The result was that consuming caffeine made blood pressure and heart rate go up for a short period,  and that levels of the hormone norepinephrine  in the blood also went up. Oh, and that consuming caffeine led to more caffeine in the bloodstream than consuming no caffeine.

The findings about blood pressure, heart rate, and norepinephrine are about as surprising as the finding about caffeine in the blood. If you do a Google search on “caffeine blood pressure”, the recommendation box at the top of the results is advice from the Mayo Clinic. It begins

Caffeine can cause a short, but dramatic increase in your blood pressure, even if you don’t have high blood pressure.

The Mayo Clinic, incidentally, is where the new experiment was done.

I looked at the PubMed research database for research on caffeine and blood pressure.  The oldest paper in English for which I could get full text was from 1981. It begins

Acute caffeine in subjects who do not normally ingest methylxanthines leads to increases in blood pressure, heart rate, plasma epinephrine, plasma norepinephrine, plasma renin activity, and urinary catecholamines.

This wasn’t news already in 1981.

Now, I don’t actually like energy drinks; I prefer my caffeine hot and bitter.  Since many energy drinks have as much caffeine as good coffee and some have almost as much sugar as apple juice, there’s probably some unsafe level of consumption, especially for kids.

What I don’t like is dressing this up as new science. The acute effects of caffeine on the cardiovascular system have been known for a long time. It seems strange to do a new human experiment just to demonstrate them again. In particular, it seems ethically dubious if you think these effects are dangerous enough to put out a press release about.


November 10, 2015

New blood pressure trial

A big randomised trial comparing strategies for treating high blood pressure has just ended early (paper, paywalled).  There’s good coverage in the New York Times, and there will probably be a lot more over the next week. It’s a relatively complicated story.

The main points:

  • Traditionally, doctors try to get your blood pressure below 140mmHg, but some people always thought lower would be better.
  • The study, funded by the US government, randomly allocated over 9000 people with high blood pressure and some other heart disease risk factor (but not diabetes) to either try to get blood pressure of 140mmHg or try to get 120mmHg.
  • A previous trial with the same targets, but in people with diabetes, had been unimpressive: the results slightly favoured more-intensive treatment, but the difference was small, and well within the variation you’d expect by chance.
  • In the new trial blood pressure targeting worked really well: the average blood pressure in the low group was 122mmHg, and in the normal group was 135.
  • Typically, people in the low group took two or three blood pressure medications, those in the normal group typically took one or two — but in both cases with quite a lot of variation.
  • There were 76 fewer ‘primary outcome events’:  heart attack, stroke, heart failure, or death from heart disease in the low BP group, and 55 fewer deaths from any cause.
  • From the beginning, the plan was to stop whenever the difference in number of ‘primary outcome events’ exceeded a specified threshold, unless there was a good reason based on the data to continue. The difference had been just barely over the threshold at the previous analysis, and they continued. In mid-September it was clearly over the threshold, and they stopped.
  • Stopping early will tend to overestimate the benefit, but the fact that they waited for one more analysis reduces this bias.

I’m surprised the benefit from extreme blood pressure reduction is so large (in a relative sense), but even more surprised that they managed to get so many healthy people to take their treatments that consistently for over three years.  As context for this, data from a US national survey in 2011-12 showed only about two-thirds of those currently taking medications for high blood pressure even get down to 140mmHg.

In an absolute sense the risk reduction is relatively small: for every thousand people on intensive blood pressure reduction — healthy people taking multiple pills, multiple times per day — they saw 12 fewer deaths and 16 fewer ‘events’.   On the other hand, the treatments are cheap and most people can find a combination without much in the way of side effects. If intensive treatment becomes standard, there will probably be more use of combination pills to make multiple drugs easier to take.

There’s one moderately worrying factor: a higher rate of kidney impairment in the low BP group (higher by a couple of percentage points). The researchers indicate that they don’t know if this is real, permanent  damage, and that more follow-up and testing of those people is needed. If it is a real problem it could be more serious in ordinary medical practice than in the obsessively-monitored trial.  This may well explain why the trial didn’t stop even earlier:  the monitoring committee would have wanted to be sure the benefits were real given the possibility of adverse effects — the sort of difficult decision that is why you have experienced, independent monitoring committees. 

November 9, 2015

To each according to his needs

There’s a fairly overblown story in the Guardian about religion and altruism

“Overall, our findings … contradict the commonsense and popular assumption that children from religious households are more altruistic and kind towards others,” said the authors of The Negative Association Between Religiousness and Children’s Altruism Across the World, published this week in Current Biology.

“More generally, they call into question whether religion is vital for moral development, supporting the idea that secularisation of moral discourse will not reduce human kindness – in fact, it will do just the opposite.”

The research found that kindergarten (update: and primary school) children from religious families scored lower on an altruism test (a version of the Dictator game).  Given ten stickers, non-religious children would give about one more away on average than religious children.


While it’s obviously true that this sort of simple moral behaviour doesn’t require religion, the cause-and-effect conclusion the story is trying to draw is stronger than the data. I’m pretty confident the people quoted approvingly wouldn’t have been as convinced by the same sort of research if it had found the opposite result.

The research does provide convincing evidence on another point, though: three-dimensional graphics are a Bad Idea.



November 3, 2015

Dogs and asthma

One News saysThe family dog or growing up on a farm could be the keys to reducing the chances of a young person suffering from asthma.

This is pretty good research. It’s obviously not a randomised experiment, but it uses the population administrative and medical data of Sweden to get a reasonable estimate of associations, and it is consistent with other population studies and has a reasonable explanation in immunology. One News gave all the relevant numbers, and got Dr Collin Brooks from Massey in as an expert. So that’s all good.

But (you knew there was a ‘but’), the population impact is smaller than the news story suggests.  That has to be the case: New Zealand, with very high asthma rates by international standards, already has fairly high dog ownership rates.  In fact, as often happens, this new study has found less benefit than earlier, smaller studies.

At current NZ asthma rates, for every extra 100 little kids who live with dogs, the research would predict that you’d prevent one or two cases of asthma. And that’s without worrying about, say, reduced housing options for households with pets.


August 30, 2015

Genetically targeted cancer treatment

Targeting cancer treatments to specific genetic variants has certainly had successes with common mutations — the most well known example must be Herceptin for an important subset of  breast cancer.  Reasonably affordable genetic sequencing has the potential for finding specific, uncommon mutations in cancers where there isn’t a standard, approved drug.

Most good ideas in medicine don’t work, of course, so it’s important to see if this genetic sequencing really helps, and how much it costs.  Ideally this would be in a randomised trial where patients are randomised to the best standard treatment or to genetically-targeted treatment. What we have so far is a comparison of disease progress for genetically-targeted treatment compared to a matched set of patients from the same clinic in previous years.  Here’s a press release, and two abstracts from a scientific conference.

In 72 out of 243 patients whose disease had progressed despite standard treatment, the researchers found a mutation that suggested the patient would benefit from some drug they wouldn’t normally have got. The median time until these patients starting getting worse again was 23 weeks; in the historical patients it was 12 weeks.

The Boston Globe has an interesting story talking to researchers and a patient (though it gets some of the details wrong).  The patient they interview had melanoma and got a drug approved for melanoma patients but only those with one specific mutation (since that’s where the drug was tested). Presumably, though the story doesn’t say, he had a different mutation in the same gene — that’s where the largest benefit of sequencing is likely to be.

An increase from 12 to 23 weeks isn’t terribly impressive, and it came at a cost of US$32000 — the abstract and press release say there wasn’t a cost increase, but that’s because they looked at cost per week, not total cost.  It’s not nothing, though; it’s probably large enough that a clinical trial makes sense and small enough that a trial is still ethical and feasible.

The Boston Globe story is one of the first products of their new health-and-medicine initiative, called “Stat“. That’s not short for “statistics;” it’s the medical slang meaning “right now”, from the Latin statum.

August 28, 2015

Trying again


This graph is from the Open Science Framework attempt to replicate 100 interesting results in experimental psychology, led by Brian Nozek and published in Science today.

About a third of the experiments got statistically significant results in the same direction as the originals.  Averaging all the experiments together,  the effect size was only half that seen originally, but the graph suggests another way to look at it.  It seems that about half the replications got basically the same result as the original, up to random variation, and about half the replications found nothing.

Ed Yong has a very good article about the project in The Atlantic. He says it’s worse than psychologists expected (but at least now they know).  It’s actually better than I would have expected — I would have guessed that the replicated effects would average quite a bit smaller than the originals.

The same thing is going to be true for a lot of small-scale experiments in other fields.

August 5, 2015

What’s in a browser language default?

Ok, so this is from Saturday and I hadn’t seen it until this morning, so perhaps it should just be left in obscurity, but:

Claims foreign buyers are increasingly snapping up Auckland houses have been further debunked, with data indicating only a fraction of visitors to a popular real estate website are Asian.

Figures released by website reveal about five per cent of all online traffic viewing Auckland property between January and April were primary speakers of an East Asian language.

Of that five per cent, only 2.8 per cent originated from outside New Zealand meaning almost half were viewing from within the country.

The problem with Labour’s analysis was that it conflated “Chinese ethnicity” and “foreign”, but at least everyone on the list had actually bought a house in Auckland, and they captured about half the purchases over a defined time period. It couldn’t say much about “foreign”, but it was at least fairly reliable on “Chinese ethnicity” and “real-estate buyer”.

This new “debunking” uses data from a real-estate website. There is no information given either about what fraction of house buyers in Auckland used the website, or about what fraction of people who used the website ended up buying a house rather than just browsing, (or about how many people have their browser’s language preferences set up correctly, since that’s what was actually measured).  Even if captured the majority of NZ real-estate buyers, it would hardly be surprising if overseas investors who primarily prefer to use non-English websites used something different.  What’s worse, if you read carefully, is they say “online traffic”: these aren’t even counts of actual people.

So far, the follow-up data sets have been even worse than Labour’s original effort. Learning more would require knowing actual residence for actual buyers of actual Auckland houses: either a large fraction over some time period or a representative sample.  Otherwise, if you have a dataset lying around that could be analysed to say something vaguely connected to the number of overseas Chinese real-estate buyers in Auckland, you might consider keeping it to yourself.

July 28, 2015

Recreational genotyping: potentially creepy?

Two stories from this morning’s Twitter (via @kristinhenry)

  • 23andMe has made available a programming interface (API) so that you can access and integrate your genetic information using apps written by other people.  Someone wrote and published code that could be used to screen users based on sex and ancestry. (Buzzfeed, FastCompany). It’s not a real threat, since apps with more than 20 users need to be reviewed by 23andMe, and since users have to agree to let the code use their data, and since Facebook knows far more about you than 23andMe, but it’s not a good look.
  • Google’s Calico project also does cheap public genotyping and is combining their DNA data (more than a million people) with family trees from This is how genetic research used to be done: since we know how DNA is inherited, connecting people with family trees deep into the past provides a lot of extra information. On the other hand, it means that if a few distantly-related people sign up for Calico genotying, Google will learn a lot about the genomes of all their relatives.

It’s too early to tell whether the people who worry about this sort of thing will end up looking prophetic or just paranoid.

July 23, 2015

Diversity is (very slightly) good for you

This isn’t in the local news, but there are stories about it in the world media: a new paper in Nature on associations between genetic diversity and various desirable characteristics.  I’m one of the authors — and so is pretty much everyone else, since this research combines analyses from over 100 cohort studies.  The Nature paper is actually the second publication in this area that I’ve worked on.  My first Auckland MSc student in Statistics, Anish Scaria, did some analysis for a different definition of genetic diversity, and that plus data from a smaller group of cohort studies was published last year.

What did we do? Humans, like most animals and many plants1, have two copies of our complete genome2. We looked at how similar these two copies were, essentially measuring small amounts of inbreeding from distant ancestors.

Each cohort study had measured a large number of binary genetic variants, ranging from 300,000 to 1,000,000. In the first paper we looked at just the proportion of variants where the two copies were the same3. In the new paper we looked at contiguous chunks of genome where all the variants were the same in the two copies, which gives a more sensitive indication of the chunks of genome being inherited from the same distant ancestor. We compared people based on the proportion of genome that was in these contiguous chunks.

The comparisons were done separately within each cohort and the associations were then averaged: obviously you would get different genetic diversity in a cohort from Iceland versus a cohort of African-Americans, and we need to make sure that sort of difference didn’t get incorporated in the analysis. Similarly, for cohorts that recruited people of different ancestries, the comparisons were done between people of the same basic ancestry and averaged.

Our first paper found that people with more difference between their two genomic copies lived (very slightly) longer on average; the new paper found that (to a very small extent) they were taller, had higher average scores on IQ tests, and had lower cholesterol. The basic direction of the results wasn’t surprising, but the lack of association for specific diseases and risk factors was — there was no sign of a difference in diabetes, for example.

Scientifically, the data provide a little bit of extra support for height and whatever IQ tests measure having been under evolutionary selection, and a bit of negative evidence on diabetes and heart disease having been under evolutionary selection in human history. And also a bit of support for the idea that you can actually get more than a hundred groups of independent and fiercely territorial academics to work together sometimes.



1. Some important crop plants, such as wheat, cabbage, and sugarcane, are insanely more complicated
2. Yes, I’m ignoring the sex chromosomes here.
3. “Homozygous” is the technical term.