Stats Chat

Twenty-two people didn’t get murdered last year. Another 576 didn’t get robbed. Some 5300 fewer people were ripped off by fraudsters. Those who say we’re drowning in a crime wave appear not to know reported crime is the lowest it’s been this century.

 A good data-based lead in to a story about community policing in the Herald today. Because crimes are much more obvious than non-crimes (I didn’t get mugged today, and my house wasn’t broken into!), it’s easy to think that crimes are increasing whether they are or not.

Two cautionary notes: we can directly observe crime rates going up or down, but the idea that we can directly observe why this is happening is a powerful cognitive illusion.  It looks as though community policing is working, but there could be other reasons for crime going down (for example, whatever is responsible in the US).  Secondly, my reaction to “those who say we’re drowning in a crime wave” is something along the lines of “Really? Who?”  All I could find on the Google were headlines that say “crime wave” when they just mean “more than one crime.”

I don’t know for sure (though I have my suspicions) whether it’s possible to rigorously demonstrate a temperature trend using data just from New Zealand.  I’m sure it’s not possible to demostrate that mammograms save lives using data just from New Zealand.  Fortunately, there’s no real need to do so in either case.

We’ve mentioned the Berkeley Earth Surface Temperature project before.  The project is run by a physicist and former climate-change skeptic, Richard Muller, and has statistical leadership from the eminent David Brillinger.  They have taken a slightly different approach to temperature analyses: they use all available temperature records and weight them for internal consistency rather than selecting a high-quality subset, and their analyses of relationships between temperature and other factors are purely statistical, not based on climate models.  It makes astonishingly little difference (except that they can get better estimates of statistical uncertainty).

Here are two graphs from their results summary page. (more…)

Stuff has a story about another sex survey, with the lead

It appears almost a given that men’s sexual fantasies will differ from those of women and now there’s a survey to prove it.

The link is to the Huffington Post. There’s another link further down the story to the Armenian Medical Network, which, to be fair, is running almost exactly the original press release, including the typos (the respondents are described as 49.6% mend and 0.4% women).

So what was this survey?  It appears to have been a serious attempt to find stuff out, conducted by psychology researcher at the University of Granada.  It hasn’t been published, which is a bad sign and makes it hard to get details, but the press release implies that it has been accepted for publication.

The survey was conducted by questionnaire, and “A number of provincial ongoing training centers, adult education centers, Granada local library and several University of Granada and Universidad Complutense schools collaborated in this study”.  That suggests it wasn’t a random sample. The extremely equal representation of men and women  suggests perhaps a quota sample — interviewers choosing people to ask, from a set of categories. That’s not ideal, but it’s better than just putting a link out on the internet, as was done by the last sex survey we examined, and as the Spanish researchers are doing for their next study according to the press release.

Anyway, what really attracted my attention when I saw the press release was the title:

The US Preventive Services Taskforce has issued another recommendation against general population cancer screening, this time for ovarian cancer.  Although the USPTF guidelines don’t have regulatory force even in the US, let alone here, they are taken seriously because they are developed by people who, generally speaking, have a clue.

In some ways ovarian cancer is an obvious target for screening: it usually isn’t caught until quite late, and earlier diagnosis could theoretically be helpful in treatment.   The current screening approach in countries where screening is done is fairly sophisticated, using a combination of ultrasound imaging and a blood test for a protein produced by ovarian tumours. Even so, a large randomised trial didn’t show any benefit, and the harm from inevitable false positives was significant, since it takes surgery to follow up suspicious findings from screening.  All that can be done at the moment is to be aware of the symptoms of ovarian cancer and get them checked out. The Listener had a good article last year on the topic.

There are some cancer screening programs that are unquestionably lifesaving, eg, mammograms for breast cancer, smear tests for cervical cancer, colonoscopy for colon cancer, and (although the cost may not be justifiable) CT scans for lung cancer in smokers.  Some others, such as melanoma screening, are plausibly beneficial and unlikely to do much harm. In general, though, population screening of healthy people for uncommon diseases has a high bar to surmount: you have to find people with the disease who are treatable now but wouldn’t be treatable if you just waited.  Even more difficult, you also have to do sufficiently little collateral damage to the people (over 99.9% for ovarian cancer) who don’t currently have the disease.

As part of Google’s mind-reading efforts, they have been working on finding connections between search targets: ‘Knowledge Graph‘, as they call it. The idea is partly to group ambiguous search results so you can choose the ones you meant, and partly to suggest other searches that you might be interested in.

For example, if you search for ‘bacon’, you could mean the 16th Century natural philosopher, the 20th century painter, the cured pig product, or actor Kevin Bacon. At the moment, I get the food as the main set of results, with Knowledge Graph side boxes suggesting the two Francis Bacons.  Kevin doesn’t show up until much further down the list.   In partial recompense, Google is advertising Knowledge Graph by providing a Bacon Number calculator, as the Herald tells us.

An actor’s Bacon Number gives their network distance from Kevin Bacon, where each link is a movie collaboration.  The same idea was had first by the mathematicians, who calculate Erdős numbers based on research collaborations with Paul Erdős. Both Bacon and Erdős are notable for the great diversity of their collaborations, not just the sheer number, so most actors and mathematicians have surprisingly small network distances from them . My Erdős number is 45(Scott, Rao, Vijayan, Erdős), and I’m not even a mathematician. Barack Obama has a Bacon Number of 2, and acting is not what he’s most famous for.   The existence of people like Erdős or Bacon makes a huge difference to the structure of a network of contacts — without them, the network tends to come apart into disconnected chunks. Worldwide connectedness is interesting for acting or mathematics, but it is more important in infectious diseases, where a few people can make a big difference to epidemic size.  There’s a nice (if somewhat old) description of some of the research on small-world networks in a magazine from the Santa Fe Institute (in the centerfold)

Back to Google again, the Bacon Number calculator is far from perfect, but the point is that, unlike the definitive Oracle of Bacon, Google gets Bacon Numbers as a by-product of a general approach rather than as a special project. Knowledge Graph also has other failures (for example, it persistently gave the wrong photograph for Australian SF author Greg Egan), but it’s another illustration of Google’s original discovery that links between pages, rather than the contents of the pages, tell you what is important.

[update: I hadn’t realised that my Erdős number decreased from 5 to 4 a few months ago]

Team Ratings for Finals Week 2

Here are the team ratings prior to Finals Week 2, along with the ratings at the start of the season. I have created a brief description of the method I use for predicting rugby games. Go to my Department home page to see this.

Performance So Far

So far there have been 196 matches played, 119 of which were correctly predicted, a success rate of 60.71%.

Here are the predictions for last week’s games.

Predictions for Finals Week 2

Here are the predictions for Finals Week 2

As you will doubtless have heard, 63 people, mostly kids and youth, have died from ‘huffing’ over the past 12 years.  That’s about the same as the number of youths who commit suicide in six months.

Suicide coverage in the media is restrained by awareness of the risks of over-publicising it.  Along those lines, the NZ Drug Foundation (on Twitter) points us to the Chief Coroner’s recommendations (page 3) for media coverage of `volatile substance abuse’

The reporting of all volatile substance abuse is recognised  as being of a highly sensitive nature. Reporting has the potential to assist in the reduction of abuse, or conversely increase the incidence by promoting use and the availability of products that may be used. Although there are no inhalant specific media guidelines, the following considerations based on those expressed by the 1985 Senate Select Committee on Volatile Fumes in Canberra, Australia may be a useful guide:
• The products subject to abuse should not be named and the methods used should not be described or depicted.
• Reports of inhalant abuse should be factual and not sensationalised or glamourised.
• The causes of volatile substance abuse are complex and varied. Reporting on deaths should not be superficial.
• Stories should include local contact details for further information or support.

The Drug Foundation say they have a site with resources, but it seems to be down at the moment (up again now).

Following up on the discussion of green coffee extract and unsupported health claims, I’d like to point out two links.

The Nightingale Collaboration is a British initiative to oppose  unsupported health claims by making complaints to various regulatory bodies.  This includes the Advertising Standards Authority, but also the professional councils that regulate various health professions.  For example, the General Chiropractic Council regulates chiropractors in the same way that the General Medical Council regulates medical doctors, and both have a duty to investigate complaints of, say, misleading advertising by their members.

In New Zealand the main resource would be the NZ Advertising Standards Authority.  They have details of the Advertising Codes (which were formulated by advertisers and media, not handed down from above), and complaint procedures, including an online complaint form.  If you honestly believe an advertiser is making an unsupported health (or environmental, or financial) claim, you can file a complaint explaining why. Your part in the process is now over. All complaints are reviewed by the Chairman, and if there is potential merit to the claim, the advertiser has to respond:

The Advertising Standards Authority’s complaints process operates on the basis that the Advertiser must provide substantiation/evidence to support the claims made in their advertising. Therefore, if a complaint is made that an advertisement is misleading or deceptive, it is the responsibility of the Advertiser to provide sufficient information to enable the Complaints Board to assess the accuracy of claims or statements made.

Last year I filed a complaint about a website selling green tea extract, and the process seemed to work.

I’m glad to see that the cost estimates for beneficiaries released yesterday have become boring  and aren’t featured news (at least on the online media sites, I haven’t checked the squashed-tree versions).  If you’re planning to spend money getting people off benefits, it’s only financially worth doing so if you don’t spend a lot more than their benefits would have cost, so you need some idea of what that is.

I had looked briefly at a similar estimate of the cost of Parliament members. It works about to about $1 million each per year: if you add up the parts of budget Votes for the Parliamentary Counsel, Parliamentary Service, and Prime Minister and Cabinet that aren’t capital expenditure (and take off the cost of printing and distributing Parliament documents to the public) and divide by the number of MPs. To get a lifetime cost you’d need data on the distribution of time in office, which looked like it would take more than ten minutes to come up with.

We need a Parliament, and we need support for people who can’t support themselves.  Aggregate costs are a useful input to calculations, but as isolated headlines they’re not helpful.  They fail to address the basic question about any number: compared to what?

Republicans and Democrats are hearing very different news about the economy:

In this example there are more possible explanations than last time:

• They really are hearing different news, because local conditions vary.  This one can’t really be true, because the geographical polarisation of voters isn’t strong enough
• They really are hearing different news, because they get it from different sources. In some ways that’s the most worrying possibility — a massive breakdown in the effectiveness of journalism.
• They are hearing the same news, but it has different implications.  For example, perhaps Republicans think the prospect of higher tax rates on income above $250,000 is bad economic news and Democrats think it is good economic news.  I don’t think this can explain such a big and recent difference.
• They are exposed to the same news, but only really hear the bits that confirm their beliefs.  Quite likely, and worrying.
• They don’t really believe what they are saying. The most positive interpretation, except if you’re in the survey business.