Stats Chat

I’m not going to make a regular habit of criticising the Herald’s Daily Pie — for a start, it only appears in the print version, which I don’t see.  Today’s one, though, illustrates a couple of issues in graph legends

The first issue is ordering. That’s almost trivial with just two values, but I actually found it distracting to have “South Island” at the top of the legend, especially when the corresponding red wedge is higher on the page than the blue wedge. I had to look twice to work out which wedge was which.  Reordering with “North Island” at the top would have helped, as would putting the labels on the pie (instead of the numbers).

Second, there’s the Note:

The total pigs number includes all other pigs such as mated gilts, baconers, porkers, and piglets still on the farm.

which comes directly from the StatsNZ table (of data from the Agricultural Production Survey). I know that, because these tables are the only place Google can find even the sub-phrase “such as mated gilts”.  In the context of the table, the note says that the “at June 30” columns for total pigs include the “Breeding sows (1-year-old and over)” given in earlier columns of the table, plus other categories that someone interested in the data would probably be familiar with. Without the earlier columns, the reaction should be “other than what?”.

Looking at the StatsNZ table you also learn the reason why “At June 30” in the title is important. The total “includes piglets still on the farm”, but not the much larger number of ex-piglets that have become part of the pork products industry: there were over 600,000 piglets weaned on NZ farms during the year, but only 287,000 pigs still on farms as of June 30.

Team Ratings for Round 1

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Predictions for Round 1

Here are the predictions for Round 1. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

There was a lot of coverage yesterday about a potential new test for pancreatic cancer. 3News covered it, as did One News (but I don’t have a link). There’s a detailed report in the Guardian, which starts out:

A simple urine test that could help detect early-stage pancreatic cancer, potentially saving hundreds of lives, has been developed by scientists.

Researchers say they have identified three proteins which give an early warning of the disease, with more than 90% accuracy.

This is progress; pancreatic cancer is one of the diseases where there genuinely is a good prospect that early detection could improve treatment. The 90% accuracy, though, doesn’t mean what you probably think it means.

Here’s a graph showing how the error rate of the test changes with the numerical threshold used for diagnosis (figure 4, panel B, from the research paper)

As you move from left to right the threshold decreases; the test is more sensitive (picks up more of the true cases), but less specific (diagnoses more people who really don’t have cancer). The area under this curve is a simple summary of test accuracy, and that’s where the 90% number came from.  At what the researchers decided was the optimal threshold, the test correctly reported 82% of early-stage pancreatic cancers, but falsely reported a positive result in 11% of healthy subjects.  These figures are from the set of people whose data was used in putting the test together; in a new set of people (“validation dataset”) the error rate was very slightly worse.

The research was done with an approximately equal number of healthy people and people with early-stage pancreatic cancer. They did it that way because that gives the most information about the test for given number of people.  It’s reasonable to hope that the area under the curve, and the sensitivity and specificity of the test will be the same in the general population. Even so, the accuracy (in the non-technical meaning of the word) won’t be.

When you give this test to people in the general population, nearly all of them will not have pancreatic cancer. I don’t have NZ data, but in the UK the current annual rate of new cases goes from 4 people out of 100,000 at age 40 to 100 out of 100,000 people 85+.   The average over all ages is 13 cases per 100,000 people per year.

If 100,000 people are given the test and 13 have early-stage pancreatic cancer, about 10 or 11 of the 13 cases will have positive tests, but so will 11,000 healthy people.  Of those who test positive, 99.9% will not have pancreatic cancer.  This might still be useful, but it’s not what most people would think of as 90% accuracy.

Team Ratings for Round 22

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 152 matches played, 88 of which were correctly predicted, a success rate of 57.9%.

Here are the predictions for last week’s games.

Predictions for Round 22

Here are the predictions for Round 22. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Ok, so this is from Saturday and I hadn’t seen it until this morning, so perhaps it should just be left in obscurity, but:

Claims foreign buyers are increasingly snapping up Auckland houses have been further debunked, with data indicating only a fraction of visitors to a popular real estate website are Asian.

Figures released by website realestate.co.nz reveal about five per cent of all online traffic viewing Auckland property between January and April were primary speakers of an East Asian language.

Of that five per cent, only 2.8 per cent originated from outside New Zealand meaning almost half were viewing from within the country.

The problem with Labour’s analysis was that it conflated “Chinese ethnicity” and “foreign”, but at least everyone on the list had actually bought a house in Auckland, and they captured about half the purchases over a defined time period. It couldn’t say much about “foreign”, but it was at least fairly reliable on “Chinese ethnicity” and “real-estate buyer”.

This new “debunking” uses data from a real-estate website. There is no information given either about what fraction of house buyers in Auckland used the website, or about what fraction of people who used the website ended up buying a house rather than just browsing, (or about how many people have their browser’s language preferences set up correctly, since that’s what was actually measured).  Even if realestate.co.nz captured the majority of NZ real-estate buyers, it would hardly be surprising if overseas investors who primarily prefer to use non-English websites used something different.  What’s worse, if you read carefully, is they say “online traffic”: these aren’t even counts of actual people.

So far, the follow-up data sets have been even worse than Labour’s original effort. Learning more would require knowing actual residence for actual buyers of actual Auckland houses: either a large fraction over some time period or a representative sample.  Otherwise, if you have a dataset lying around that could be analysed to say something vaguely connected to the number of overseas Chinese real-estate buyers in Auckland, you might consider keeping it to yourself.

We (already!) have a Stat of the Week nomination, for the Herald’s

Conservative estimates are that the average man will lose 12,547 umbrellas during his lifetime.

At first sight this looks like deadpan humour rather than a statistic, but in that case you’d expect a by-line on the piece.  The Google, which knows all and tells all, finds thirty other examples of the statistic.  It looks as though the primary source is the Telegraph

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday August 7 2015.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of August 1 – 7 2015 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

From a Retail New Zealand Media Fact Kit

The amount New Zealanders spend on goods from foreign websites is approaching $1.5 billion and this number is growing all the time.

GST is 15%, so the total GST payable on an amount approaching $1.5 billion would be approaching $225 million.

Since $1.5 billion would be approaching $900 per household (1.68 million households according to StatsNZ), I assume it includes quite a few business purchases as well. For these, any increase in GST on the purchase would later be deducted from the business’s GST liability, leaving a net zero.

For non-business purchases, some of that GST is already payable under current law (if the total value of a single package is over $400). Some would still not be payable if the threshold was lowered to $25.  Also, some might not be payable and definitely would not be easily collectable at the border because the purchase is an electronic download — e-books or music, for example.

Putting all these together, the potential increase in revenue has to be less than 15% of $1.5 billion, though it’s hard to say how much less.

The Media Fact Kit says

The Government misses out on at least $200 million in tax (maybe as much as $500 million) every year

Based on their expenditure numbers, that doesn’t look plausible. Still, maybe their numbers are wrong and it really is more than $200 million, and maybe even as much as $500 million.  GST would still cost something to collect, and more than it does (per dollar) within New Zealand, so the net gain to the Government would be noticeably smaller than this. They don’t even pretend to take the costs into account, they just talk about how many hip replacements could be funded with all the free money.

One of the differences between a Treasury Regulatory Impact Statement and a “Fact Kit” from a lobby group is that the numbers in the RIS have to add up and the document needs to give sources. If, as Radio New Zealand reports, there will be a proposal for Cabinet this month, we might be able to get some real numbers about the likely revenue and costs, and perhaps even how the economic impact would compare to other ways of raising taxes.

A year-old pie chart describing Google+ users. On the right are two slices that would make up a valid but pointless pie chart: their denominator is Google+ users. On the left, two slices that have completely different denominators: all marketers and all Fortune Global 100 companies.

On top of that, it’s unlikely that the yellow slice is correct, since it’s not clear what the relevant denominator even is. And, of course, though most of the marketers probably identify as male or female, it’s not clear how the Fortune Global 100 Companies would report their gender.

From @NoahSlater, via @LewSOS, originally from kwikturnmedia about 18 months ago.

You’ve probably heard that there are positive results from an Ebola vaccine trial (3News, Radio NZ, Stuff, Herald). The stories are all actually good. Here’s the (open-access) research paper

The vaccine was genetically engineered: it’s a live virus for an animal disease that doesn’t spread in humans, modified to produce just one Ebola protein. Having a live virus makes the immune system respond more enthusiastically, but you wouldn’t want to risk a vaccine containing anything even remotely like live Ebola virus. Genetic engineering produces a live virus that contains none of the functional bits of Ebola, so that even if it (improbably ) turned out to be able to spread, it wouldn’t be a big deal.

The basic trial design was to find Ebola cases and vaccinate their contacts and the contacts of their contacts, with randomisation between immediate vaccination and vaccination 21 days later.  The design was a good compromise: the public-health authorities need to know if the vaccine works in order to decide whether it can be used to control future epidemics, but since it probably does little harm, most people would want to be vaccinated.  With this design, everyone the doctors talk to will get the vaccine, either immediately or in three weeks. The design is also cost-effective, since everyone you need to vaccinate is someone the public health system would want to check up on anyway.

In practice, not everyone eligible will be end up being vaccinated: some will refuse, and some will not be contactable. You have to decide how to include the unvaccinated people in the analysis.  In this trial, no-one in the immediate-vaccination group who actually got vaccinated ended up with Ebola, which is what gives the headline 100% success rate. If you compare people who were randomised to immediate vaccination, whether they got it or not, with those who were randomised to delayed vaccination (a more common analysis strategy), the vaccine was still estimated as 75% effective.

There’s still some work to do: when the vaccine is used, it will be important to keep track as far as possible of how well it works. That’s important because we need to know if it’s worth working on a new vaccine, or whether to divert resources to research on treatments for those who are infected, or to other diseases.  For a change, though, this is good news.