Stats Chat

…”One of the problems in developing drugs is detecting serious side effects. People who need medication tend to be unwell, so it’s hard to find a reliable comparison. That’s why the roughly threefold increase in heart-attack risk among Vioxx users took so long to be detected …”

Read his column, Faulty Powers, here.

Team Ratings for Round 5

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 32 matches played, 14 of which were correctly predicted, a success rate of 43.8%.

Here are the predictions for last week’s games.

Predictions for Round 5

Here are the predictions for Round 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 8

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 42 matches played, 26 of which were correctly predicted, a success rate of 61.9%.

Here are the predictions for last week’s games.

Predictions for Round 8

Here are the predictions for Round 8. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

From Fox News last week (via)

My edit based on what ended up happening

If the magnitudes don’t matter, the graph can’t be worth the pixels it’s printed on.

Statistics New Zealand has just released the meshblock-level data from last year’s Census, together with matching information for the previous two censuses (reworked to use the new meshblock boundaries).

Mashblock shows one thing that can be built with this sort of data, there are many others.

Get your meshblock files here

There’s an interesting piece in Stuff about Massey’s Illegal Drug Monitoring System. I’d like to make two points about it.

First, the headline is that synthetic cannabis use is declining. That’s good, but it’s in a survey of frequent users of illegal drugs.  If you have the contacts and willingness to buy illegal drugs, it isn’t surprising that you’d prefer real cannabis to the synthetics — there seems to be pretty universal agreement that the synthetics are less pleasant and more dangerous.  This survey won’t pick up trends in more widespread casual use, or in use by teenagers, which are probably more important.

Second, the study describes the problems caused by much more toxic new substitutes for Ecstacy and LSD. This is one of the arguments for legalisation. On the other hand, they are also finding increased abuse of prescription oxycodone. This phenomenon, much more severe in the US, weakens the legalisation argument somewhat.  Many people (including me) used to believe, based on reasonable evidence, that a substantial fraction of the adverse health impact of opioid addiction was due to the low and unpredictably-varying purity of street drugs, and that pure, standardised drugs would reduce overdoses. As Keith Humphreys describes, this turns out not to be the case.

Just a quick pointer to a nice opinion piece by the Financial Times’ “Undercover Economist” and star of BBC Radio 4’s excellent “More or Less” podcast, Tim Harford. Tim very nicely argues that in the hype over big data, stories of the failures of simplistic, correlation driven approaches rarely get airtime, and hence we get a misleading impression about the efficacy of these techniques.

The DimPost posted a new poll average and trend, which gives an opportunity to talk about some of the issues in interpretation (you should also listen to Sunday’s Mediawatch episode)

The basic chart looks like this

The scatter of points around the trend line shows the sampling uncertainty.  The fact that the blue dots are above the line and the black dots are below the line is important, and is one of the limitations of NZ polls.  At the last election, NZ First did better, and National did worse, than in the polling just before the election. The trend estimates basically assume that this discrepancy will keep going in the future.  The alternative, since we’ve basically got just one election to work with, is to assume it was just a one-off fluke and tells us nothing.

We can’t distinguish these options empirically just from the poll results, but we can think about various possible explanations, some of which could be disproved by additional evidence.  One possibility is that there was a spike in NZ First popularity at the expense of National right at the election, because of Winston Peters’s reaction to the teapot affair.  Another possibility is that landline telephone polls systematically undersample NZ First voters. Another is that people are less likely to tell the truth about being NZ First voters (perhaps because of media bias against Winston or something).  In the US there are so many elections and so many polls that it’s possible to estimate differences between elections and polls, separately for different polling companies, and see how fast they change over time. It’s harder here. (update: Danyl Mclauchlan points me to this useful post by Gavin White)

You can see some things about different polling companies. For example, in the graph below, the large red circles are the Herald-Digipoll results. These seem a bit more variable than the others (they do have a slightly smaller sample size) but they don’t seem biased relative to the other polls.  If you click on the image you’ll get the interactive version. This is the trend without bias correction, so the points scatter symmetrically around the trend lines but the trend misses the election result for National and NZ First.

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday April 4 2014.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of March 29 – April 4 2014 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

I saw this story in the Herald and didn’t read it in detail, just thought it was an interesting calculation to do

The Financial Times reported last week that the average global price of eight breakfast staples had risen almost 25 per cent this year.

The increases mainly affected coffee, orange juice, wheat, sugar, milk, butter, cocoa and pork.

We decided to create a Kiwi version of the Financial Times story and Statistics NZ food price figures reveal New Zealand families are not exempt from the trend.

David Farrar did read the story, and so was rather less impressed, as he also mentioned on Twitter.  The problem is that the calculation was done wrong.

If you served tomatoes, mushrooms, bacon, toast, eggs, tinned spaghetti and cereal, with coffee, tea and orange juice this weekend, it would have cost you 6.9 per cent more than the same meal in 2008, and almost 3 per cent more than in 2012. Breakfast food prices have risen more quickly than other prices.

Over the past five years, the compound average annual rate of inflation was 2.1 per cent.

If the average annual rate was 2.1%, which sounds about right, the total increase over five years would be 2.1% five times, which turns out to be 11%. Since 6.9% is less than 11%, breakfast food prices have risen less quickly than other prices. Quite a bit less. The story has it completely backward.

If you’re reading especially carefully, you might also notice that it’s more than five years from “this weekend” back to 2008 — for example, a comparison of end of March 2008 to this weekend would be a six year period.

This is the sort of thing that a subeditor should spot. It’s also the sort of thing the RBNZ inflation calculator is useful for — you put in a number and two years and it does the calculations.  If you use the calculator, you find that “this weekend” is apparently December 2013, and the 2008 comparison is December 2008, rather than March 2008 to March 2013. You’d also see that the sub-index for food had increased less than the total CPI, which would presumably make you more suspicious about the story.

There’s also some discussion of individual item prices. This doesn’t have the awful 5:1 error ratio of the main argument, but it still demonstrates where a bit of thinking could have helped

Mild Arabica coffee was trading on the commodity markets for US$1.76 ($2.03) a pound (453g) in February, up from US$1.35 in January. Mild Arabica coffee was trading on the commodity markets for US$1.76 ($2.03) a pound (453g) in February, up from US$1.35 in January.

If you go to the Countdown website you find that their Signature range coffee beans cost NZ$6 for 200g, or roughly US$12 per pound. Obviously most of the cost is not the wholesale commodity price. That’s presumably even more true for instant coffee (the authentic version of the beverage in a ‘traditional cooked breakfast’)

The components of the CPI that have increased fastest aren’t all that surprising if you read the Herald regularly. For example, the cost of home ownership was up 27% over that five-year period, insurance was up 26%, education, and cigarettes and tobacco were up 67%.

If some things go up faster than average, others must go up slower or even decrease. Household appliances and furniture are down a bit. Telecommunications equipment, computing equipment,  and telecommunications services have gotten much cheaper. You can hardly give away a 2008 phone or computer (though if you’re trying to, Te Whare Marama refuge will put it, and more recent kit, to good use)