All decision rules based on observed data make mistakes, whether they are detecting the Higgs Boson or breast cancer.  And the more resources you can invest in each decision, the lower you can (typically) push the error rate.

Copyright enforcement has been a dramatic example of the opposite trend.  A century ago, individuals basically couldn’t infringe copyright in any meaningful way, so the occasional disputes got settled by expensive lawsuits.  In the modern world, copying is easy, so the companies that own lots of copyrights are looking for cheap and dirty ways to detect and stop copying.  Of course, the error rate goes up.

A dramatic example happened last night, when the live video feed for the Hugo Awards ceremony at the World Science Fiction Convention was shut down  by automated filtering systems at the company doing the broadcasting. One of the awards, for “Best Dramatic Presentation, short form” was being given for a Doctor Who script.  Clips from the TV show were shown — with full permission of the copyright owners, not that permission would have been necessary for this purpose — and some computer program made a false positive decision.

[Update: the same thing happened, briefly, to the YouTube feed of the Democratic National Convention later the same week]

The Herald has a story about junk food being linked to dementia.  The story itself is fine, but the sources are interesting:

Too much fatty and sweet food can increase insulin levels causing fat, muscle and liver cells to stop responding to the hormone, Medical Daily explained.

When the brain stops responding to insulin our capacity to think and create new memories is hindered, according to the paper recently published in New Scientist magazine.

 Medical Daily is an aggregator of medical stories — it’s a good place to find interesting research with lively descriptions, but it’s not very selective.    It’s a very popular source for the mass media and for alternative-medicine blogs.   The unusual aspect of the story is the primary source, New Scientist.   New Scientist is not a research journal, it’s a popular science magazine.  It used to be the best source in the world for accessible, detailed, science journalism, and although it’s become a bit sensationalist, there’s still a lot of good material there.   It has never been a primary source, and it hasn’t suddenly started publishing new research.

In fact, the New Scientist article is well worth reading: it describes the theory that some (most?) Alzheimer’s disease is due to insulin resistance in the brain, with quotes from researchers and links to the original research.  The “Type III diabetes” theory of dementia is definitely a minority view, and the evidence for it is thought-provoking, but not definitive.  There doesn’t seem to be any reason why it couldn’t be at least partly true, but the same could have been said for the aluminium theory of dementia in its early days.

The only good thing about the diabetes theory of dementia is that while we’re not great at treating insulin resistance, we’re a lot better at it than we are at treating Alzheimer’s.

That’s a five-set Venn diagram, invented in 1975.  Some mathematicians have just worked out how to do an 11-set one.

These are pretty, and a computational and mathematical achievement, but unfortunately they don’t have any of the properties that make Venn diagrams a marginally-useful visualization method.     (via)

Andrew Gelman, of Columbia University, wrote an article a few months ago called “Statistics for Cigarette Sellers”

Remember How to Lie with Statistics? It turns out the  author worked for the cigarette companies… It appears he was also working on a book in the late 1960s called How to Lie with Smoking Statistics, which the publisher saw “high likelihood of proceeding into print.”

His blog has a largely overlapping post, but with a bit more material (and discussion in comments).

An annoying trope in US political stories (which, fortunately, hasn’t made it here yet) is to criticise a politician for using too many first person singular pronouns — saying “I” and “me” too often.  These stories often get analysed by the computational linguists over on Language Log, who tiredly point out that the claim is (a) largely irrelevant to the point being made, and (b) false.

This week’s example is the speech by New Jersey Governor Chris Cristie at the Republican National Convention.  A New York Times piece said

Gov. Chris Christie is getting rave reviews today for his performance at the National Republican Convention, and there’s no doubt in my mind that he did a huge amount of good for the three most important people in his life – he, himself, and him….

By my count, Mr. Christie used the word “Romney” six times in his address. He used the word “I” 30 times, plus a couple of “me’s” and “my’s” tossed in for seasoning.

but as Mark Liberman points out, the actual use of first person singular pronouns was slightly lower than in the speeches by Paul Ryan, Anne Romney, Rick Santorum, Mike Huckabee, and much lower than Clint Eastwood.  It’s also slightly lower than in the similar speech by (then Illinois Senator) Barack Obama at the 2004 Democratic Convention.

Since newspapers almost exclusively cover positive results in medical research, it’s easy to have an unrealistically optimistic view of progress.  Most weeks, there are multiple stories about some new compound or extract that kills cancer cells in the lab, but we don’t learn what happens later.  And no, these treatments aren’t suppressed by evil multinational drug companies, who are actually in desperate need of new things to sell us.  As a partial corrective, here are some relatively recent research findings that make it less likely we’ll soon get new treatments for important diseases.

Alzheimer’s:  trials of two antibodies against amyloid plaques, which had shown promise in mice, failed to  show benefits in patients (one was robustly null, the other was borderline and the company hasn’t quite given up hope).  Inhibiting one of the enzymes that makes amyloid has also failed.

Heart disease:  Raising ‘good’ HDL cholesterol was a big hope, but three separate approaches to treatment have so far failed to provide any convincing benefit in clinical trials, and genetic studies suggest that although HDL is associated with lower risk, it may not actually be responsible for the reduction.

 Diabetes:  drugs to reduce insulin resistance through a new target called PPAR were going to be the new revoluation.  Troglitazone and rosiglitazone (Avandia) got into use but turned out not to actually reduce complications of diabetes, several others didn’t make it past clinical trials.  An even more promising approach was to inhibit one of the transporter proteins in the kidney, and dump all the excess glucose in urine. It didn’t work either.

One piece of good news as balance: a very promising new antibiotic for tuberculosis has succeeded in a phase II trial of drug-resistant TB.  The trial looked at how fast patients became non-contagious, and the new antibiotic got more patients to that state, faster.  The next step, just about to start, is a longer trial to see if patients are actually cured — it could still fail, of course.

One of the problems in medical research, exacerbated by the new ability to measure millions of genetic variables at once, is that you can always divide people into sensible subgroups.

If your treatment doesn’t work, or your hated junk food isn’t related to cancer, overall, you can see if the relationship is there in men or in women.  Or in younger or older people.  Or in Portugese-speaking bassoonists. The more you chop up the data, the more likely you are to find some group where there’s a difference.  You can then focus on that group in your results.

To combat this tendency, my Seattle colleague Noel Weiss has been promoting the slogan “conclusions of difference require evidence of difference”.  That is, if you want to report that cupcakes cause cancer in men but not in women, you need evidence that the relationship is different in men and in women.  Finding supportive evidence in men but not finding it in women isn’t enough: that’s not evidence of a difference.  Needing evidence of a difference is especially important when you wouldn’t expect a difference.  We expect most things to have basically similar effects in men and women, and where the effects are different there’s usually an obvious reason.

All this is leading up to a story in the Herald, where a group of genetics researchers claim that a well-studied variant in a gene called monoamine oxidase increases happiness in women, but not in men.  We know this is surprising, because the researcher said so — they were expecting a decrease in happiness, and they don’t seem to have been expecting a male:female difference.  The researchers say that the difference could be because of testosterone — and of course it could be, but they don’t present any evidence at all that it is.

Anyway, as you will be expecting by now, I found the paper (the Herald gets points for giving the journal name), and it is possible to do a simple test for differences in `happiness’ effect between men and women. And there isn’t much evidence for a difference. For people who collect p-values: about 0.09 (Bayesian would get a similar conclusion after a lot more work). So, if we didn’t expect a benefit in  women and no difference in men, the data don’t give us much encouragement for believing that.

Testing for differences isn’t the ideal solution — even better would be to fit a model that allows for a smooth variation between constant effect and separate effect — but testing for differences is a good precursor to putting out a press release about differences and trying for headlines all over the world. We can’t expect newspapers to weed this sort of thing out if scientists are encouraging it via press releases.

Last Wednesday I wrote that a survey should always provoke the questions “How was it done? By whom? and What are they selling?”

This week, Stuff and TVNZ tell us that half of New Zealanders don’t have a will, and that there are more wills in the South Island than the North Island.  This survey was commissioned by Public Trust (who do wills), but there’s no information about how the survey was done or by whom.  Public Trust don’t currently have any information on their ‘News‘ webpage.

An interesting coincidence, but probably no more: at the moment, the proportion saying they have a will on the clicky poll on the Stuff website is 47%.

Stuff has an example of the recurring problem of an perfectly good medical science story spoiled by not one, but two appalling headlines.

Currently, on the ‘Wellbeing’ section of the website you see

Feeling poorly? Try Vegemite

and on clicking through to the story

Vegemite may ward off superbugs

The story then goes on to say that high concentrations of niacin, a vitamin found in many foods, including, yes, Vegemite, has been found in test-tube and mouse studies to help in killing bacteria.  It then says

The Internets are full of a story about herbal tea curing breast cancer. The Herald is actually more restrained than many:

An extract commonly found in herbal tea can stop the spread of breast cancer, researchers have found.

That isn’t what the researchers found.  For a start, unless you’re in or near Pakistan, the extract isn’t commonly found in herbal tea.  But that’s not the main problem.