Posts written by Thomas Lumley (2645)

From the Twitters

Some people are suggesting the 99.9% bus punctuality stats might be made up. I am shocked. Truly shocked.

— AkTransportBlog (@AkTransportBlog) August 21, 2013

There followed a lively discussion, with comparisons to election results in various unpleasant places.

As I’ve suggested before, the really damning aspect of the 99.9% bus punctuality stats is that they might actually be honest. That is, `punctuality’ has been defined in a way that it does not reflect passenger experience and is of almost no interest to passengers. A bus is ‘punctual’ if it starts its route (which I assume means the driver starts up the ticket system) no more than 5 minutes late. What happens later doesn’t matter, as long as the bus does eventually reach its destination.

The real-time prediction system (although it may not be great at predicting the future) knows where the buses have been, so it would be feasible to set up a punctuality summary that actually measured punctuality — perhaps the proportion of major timepoints where the bus was no more than 5 minutes late or 1 minute early in departing.

I mention this point again because I’m teaching Design of Clinical Trials at the moment, and one of the big issues is ‘surrogate outcomes’. That is, people running studies are often tempted to measure something convenient rather than what patients care about: diabetes trials may measure blood sugar rather than heart attacks and kidney failure; heart disease trials may measure blood pressure rather than heart attacks or strokes; cancer trials may measure tumour size rather than survival or symptoms.  This matters, because there are important examples where a treatment improves a surrogate outcome, but makes the real outcomes worse.

There is no way that the Auckland bus punctuality statistics are accurate measurements of something that matters to passengers. It’s not a matter of life and death as it is in clinical trials, but it’s still a waste of time and money.  And since the Auckland bus system is actually pretty good, it makes Auckland Transport and the Council look unnecessarily stupid.

The US Census Bureau has put together an interactive map of languages other than English spoken in the US — you can look at all speakers of the language, or just those who are not fluent in English, for a range of languages.

Here’s the map for (preferentially) French speakers, showing concentrations in Maine (near French Canada) and Louisiana (Cajun) that I’d expect, groups in the major cities, and a cluster I don’t understand in northwestern Georgia

Diversity edition

• Statistical classification (data mining, learning) algorithms will learn to match your classifications. Sometimes that’s not quite what you want
• People are starting to comment more on entirely-male conference speaker lineups. Here’s a simple calculator for the probability distribution of number of people from some group (eg women), for a given proportion in the target population. For example, if you select 10 people from a population that’s 15% female, you should have only a 1/9 chance of not getting any women.
• Map of scientific collaborations, based on co-authorship of books and papers. (click for a really big version)

Once again, the Herald is relying on sympathetic magic in a nutrition story (previous examples)

1. Walnuts: These nuts look just like a brain, so it makes sense that they’re packed with good stuff for your grey matter.The British Journal of Nutrition reported that eating half a cup of walnuts a day for eight weeks increased reasoning skills by nearly 12 per cent in students. 

There’s no way that the appearance of a natural food could possibly be a guide to its nutritional value — how would the walnut know that it’s good for human brains, and why would it care? Pecans, which look a bit like brains, don’t contain the levels of n-3 fatty acids that are supposed to be the beneficial component of walnuts, and fish and flax seeds, which do contain n-3 fatty acids, don’t look like brains.

The story gets two cheers for almost providing a reference: searching on “British Journal of Nutrition walnuts reasoning skills” leads to the paper. It’s a reasonable placebo-controlled randomised experiment, with participants eating banana bread with or without walnuts.  The main problem is that the researchers tested 34 measurements of cognitive function or mood, and found a difference in just one of them.  As they admit

The authors are unable to explain why inference alone was affected by consumption of walnuts and not the other ‘critical thinking’ subtests – recognition of assumption, deduction, interpretation, and evaluation of arguments.

The prior research summarised in the paper shows the same problem, eg,  one dose of walnuts improved one coordination test in rats, but a higher dose improved a different test, and the highest dose didn’t improve anything.

There’s an interesting piece on cannabis risks at Project Syndicate. One of the things they look at is the correlation between frequent cannabis use and psychosis.  Many people are, quite rightly, unimpressed with the sort of correlation, since it isn’t hard to come up with explanations for psychosis causing cannabis use or for other factors causing both.

However, there is also some genetic data.  The added risk of psychosis seems to be confined to people with two copies of a particular genetic variant in a gene called AKT1. This is harder to explain as confounding (assuming the genetics has been done right), and is one of the things genetics is useful for. This isn’t just a one-off finding; it was found in one study and replicated in another.

On the other hand, the gene AKT1 doesn’t seem to be very active in brain cells, making it more likely that the finding is just a coincidence.  This is one of the things bioinformatics is good for.

In times like these it’s good to remember Ben Goldacre’s slogan “I think you’ll find it’s a bit more complicated than that.”

TV3 has tried to stir up the issue of the death penalty in New Zealand.  They have a poll showing majority opposition by the country as a whole, and by supporters of every party except NZ First.  Even the Sensible Sentencing Trust isn’t in favor.

The lead-in to the story is that the murder rate has never ‘recovered’ from the abolition of the death penalty.  They have a graph showing homicides per capita rising and then falling again, but not to the earlier levels.

Using the term ‘recovered’ comes very close to asserting a causal connection; but is there even a reliable correlation? International comparisons are useful here.  I don’t have long time series for homicide, but Kieran Healy has produced a graph of international trends in deaths due to assault. This isn’t the same as homicide, but is close enough to be relevant.

Here’s the New Zealand panel, with the arrow indicating the abolition of the death penalty. The details are slightly different from those for homicide, but the basic trend is the same that TV3 reports.

and here’s the international comparison, with NZ second from the bottom, on the left. As usual, click to embiggen

The NZ pattern is very similar to other countries, including Australia (where abolition didn’t happen until about 10 years later), Finland (where it was abolished in 1949 for crimes committed in peacetime), and Switzerland (1942).

If you look at the countries that still have the death penalty, murder rates are low and falling in Japan, South Korea had the same sort of rise and fall that NZ has had (over a shorter time scale), and of course there’s the USA.

It doesn’t look as though the death penalty is a major driving force in these patterns.

From the Guardian, a story about a barchart in the Sun portraying the cost of ‘green energy’

From a number of fields

• The WiFi system at the British Library wouldn’t let a researcher get to an online copy of Hamlet, because of the violent content. “The site is dismal, and our affairs from England come too late”
• From one of the Guardian’s science blogs: [update: link fixed]

So when one particular paper began to strain the servers, attracting hundreds if not thousands of downloads, the entire editorial board began to pay attention. “What,” they asked, “is so special about this paper on the ryanodine receptor of Caenorhabditis elegans?” (For those of you who don’t know, Caenorhabditis elegans is a very common and much-loved model animal—it’s a small, soil-living roundworm with some very useful features. Please don’t ask me what a ryanodine receptor is; I don’t know and I don’t really care.)

• Along similar lines, someone reminded me of the problem the UK town of Scunthorpe has with text filtering.  There is an old joke that there are two other football teams whose names contain swear words (punchline)

Nate Silver’s talk at the Joint Statistical Meetings a couple of weeks ago identified groupthink by insiders as a major threat to accurate journalism.  It’s natural to believe the opinions of other intelligent, well-informed people, but if they are working from  the same limited set of information as you, the fact that they agree with you is not actually good evidence that you’re right.

Tim Harford has a column this week arguing that diversity is important, for basically this reason. Even if, say, women or Maori or Asians or small-town farm boys aren’t actually better at decision-making, the simple fact that they have different starting points can lead to better decision-making by groups. It doesn’t have to, but it can. The New York Times also has a piece on this theme, by political scientist Scott Page.

An important empirical basis for the idea that diversity is, in itself, beneficial is the work of Solomon Asch on conformity.  Asch found that even in very simple decisions, many people would follow an obviously-wrong choice made by enough other people. On the other hand, even a single non-conforming voice, and even one that was more wrong than the consensus, could free people to follow the evidence of their own eyes.

XKCD, reminding us of the difference