June 7, 2017

Super 18 Predictions for Round 16 Game, Hurricanes vs Chiefs

Team Ratings for Round 16 Game, Hurricanes vs Chiefs

The basic method is described on my Department home page.

This week is pretty crazy, just one game from round 16 when round 15 has not been completed and won’t be for a month.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Current Rating Rating at Season Start Difference
Hurricanes 17.92 13.22 4.70
Crusaders 13.98 8.75 5.20
Highlanders 11.43 9.17 2.30
Lions 10.96 7.64 3.30
Chiefs 8.49 9.75 -1.30
Brumbies 3.44 3.83 -0.40
Blues 2.65 -1.07 3.70
Sharks 1.52 0.42 1.10
Stormers 0.53 1.51 -1.00
Waratahs -0.50 5.81 -6.30
Bulls -5.20 0.29 -5.50
Jaguares -5.38 -4.36 -1.00
Force -8.85 -9.45 0.60
Cheetahs -9.83 -7.36 -2.50
Reds -10.78 -10.28 -0.50
Kings -13.53 -19.02 5.50
Rebels -15.58 -8.17 -7.40
Sunwolves -18.38 -17.76 -0.60

 

Performance So Far

So far there have been 120 matches played, 91 of which were correctly predicted, a success rate of 75.8%.
Here are the predictions for last week’s games.

Game Date Score Prediction Correct
1 Blues vs. Reds Jun 02 34 – 29 14.60 TRUE
2 Crusaders vs. Highlanders Jun 03 25 – 22 6.50 TRUE
3 Chiefs vs. Waratahs Jun 03 46 – 31 12.70 TRUE
4 Brumbies vs. Rebels Jun 03 32 – 3 21.60 TRUE
5 Force vs. Hurricanes Jun 03 12 – 34 -22.90 TRUE

 

Predictions for Round 16, Hurricanes vs. Chiefs

Here are the predictions for the Round 16 game this week. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Game Date Winner Prediction
1 Hurricanes vs. Chiefs Jun 09 Hurricanes 12.90

 

NRL Predictions for Round 14

Team Ratings for Round 14

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Current Rating Rating at Season Start Difference
Storm 7.50 8.49 -1.00
Broncos 5.57 4.36 1.20
Sharks 5.18 5.84 -0.70
Raiders 5.17 9.94 -4.80
Panthers 3.37 6.08 -2.70
Sea Eagles 3.19 -2.98 6.20
Roosters 3.00 -1.17 4.20
Cowboys 2.07 6.90 -4.80
Dragons 0.73 -7.74 8.50
Eels -1.60 -0.81 -0.80
Titans -2.11 -0.98 -1.10
Warriors -3.71 -6.02 2.30
Rabbitohs -4.69 -1.82 -2.90
Bulldogs -5.04 -1.34 -3.70
Wests Tigers -7.57 -3.89 -3.70
Knights -13.12 -16.94 3.80

 

Performance So Far

So far there have been 99 matches played, 58 of which were correctly predicted, a success rate of 58.6%.
Here are the predictions for last week’s games.

Game Date Score Prediction Correct
1 Storm vs. Knights Jun 02 40 – 12 23.30 TRUE
2 Eels vs. Warriors Jun 02 32 – 24 5.70 TRUE
3 Dragons vs. Wests Tigers Jun 03 16 – 12 13.30 TRUE
4 Roosters vs. Broncos Jun 03 18 – 16 0.70 TRUE
5 Cowboys vs. Titans Jun 03 20 – 8 6.80 TRUE
6 Sea Eagles vs. Raiders Jun 04 21 – 20 1.60 TRUE
7 Bulldogs vs. Panthers Jun 04 0 – 38 0.90 FALSE

 

Predictions for Round 14

Here are the predictions for Round 14. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Game Date Winner Prediction
1 Sharks vs. Storm Jun 08 Sharks 1.20
2 Sea Eagles vs. Knights Jun 09 Sea Eagles 19.80
3 Broncos vs. Rabbitohs Jun 09 Broncos 13.80
4 Titans vs. Warriors Jun 10 Titans 5.60
5 Panthers vs. Raiders Jun 10 Panthers 1.70
6 Eels vs. Cowboys Jun 10 Cowboys -0.20
7 Wests Tigers vs. Roosters Jun 11 Roosters -7.10
8 Bulldogs vs. Dragons Jun 12 Dragons -2.30

June 5, 2017

Briefly

  • Possibly a record false positive rate:  “a substantial number of takedown requests submitted to Google are for URLs that have never been in our search index, and therefore could never have appeared in our search results… Nor is this problem limited to one submitter: in total, 99.95% of all URLs processed from our Trusted Copyright Removal Program in January 2017 were not in our index” (Google submission to Register of Copyrights(PDF), via Techdirt)
  • Problem with rental costs in Canada’s historical CPI “the clerks who recorded the data were under an instruction that, since the CPI was to represent prices paid by better off working class families, to edit out any rental figures what were above a designated threshold. By the end of the 1950s they were throwing out more than half of the reported rents.” (Worthwhile Canadian Initiative). Data doesn’t just happen: it’s choices by people.
  • I’ve mentioned the University of Washington course “Calling Bullshit on Big Data” before. Now the New Yorker has a story about it.
  • What different sorts of things can go wrong with a statistical prediction rule? A taxonomy, from Ed Felten.
  • Explore NZ mortality rates divided up by ethnicity, income, and age
  • “What we learned from three years of interviews with data journalists, web developers and interactive editors at leading digital newsrooms” Storybench, via Alberto Cairo
  • A couple of examples from the fine UK election tradition of disinformation graphics: Scotland, London

Stat of the Week Competition: June 3 – 9 2017

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

  • Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday June 9 2017.
  • Statistics can be bad, exemplary or fascinating.
  • The statistic must be in the NZ media during the period of June 3 – 9 2017 inclusive.
  • Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

June 2, 2017

Time for stakeholder participation?

Q: Did you see `young blood’ cuts cancer and Alzheimer’s risk?

A: That’s the headline, yes.

Q: This is the Silicon Valley startup that’s transfusing young people’s blood into older people?

A: Well, Monterey rather than Silicon Valley, but yes.

Q: Isn’t it a pity we used up all the vampire jokes on Theranos?

A: I’m sure they aren’t really dead, just sleeping. (more…)

May 30, 2017

Super 18 Predictions for Round 15

Team Ratings for Round 15

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Current Rating Rating at Season Start Difference
Hurricanes 17.97 13.22 4.70
Crusaders 14.19 8.75 5.40
Highlanders 11.22 9.17 2.10
Lions 9.87 7.64 2.20
Chiefs 8.35 9.75 -1.40
Blues 3.23 -1.07 4.30
Brumbies 3.00 3.83 -0.80
Sharks 1.52 0.42 1.10
Stormers 0.53 1.51 -1.00
Waratahs -0.36 5.81 -6.20
Bulls -5.20 0.29 -5.50
Jaguares -5.38 -4.36 -1.00
Force -8.90 -9.45 0.60
Cheetahs -9.83 -7.36 -2.50
Reds -11.35 -10.28 -1.10
Kings -12.44 -19.02 6.60
Rebels -15.14 -8.17 -7.00
Sunwolves -18.38 -17.76 -0.60

 

Performance So Far

So far there have been 111 matches played, 85 of which were correctly predicted, a success rate of 76.6%.
Here are the predictions for last week’s games.

Game Date Score Prediction Correct
1 Blues vs. Chiefs May 26 16 – 16 -1.80 FALSE
2 Reds vs. Force May 26 26 – 40 3.10 FALSE
3 Sunwolves vs. Cheetahs May 27 17 – 38 -2.30 TRUE
4 Highlanders vs. Waratahs May 27 44 – 28 15.50 TRUE
5 Rebels vs. Crusaders May 27 19 – 41 -25.80 TRUE
6 Bulls vs. Hurricanes May 27 20 – 34 -19.90 TRUE
7 Sharks vs. Stormers May 27 22 – 10 3.50 TRUE
8 Jaguares vs. Brumbies May 27 15 – 39 -1.70 TRUE

 

Predictions for Round 15

Here are the predictions for Round 15. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Game Date Winner Prediction
1 Blues vs. Reds Jun 02 Blues 18.60
2 Crusaders vs. Highlanders Jun 03 Crusaders 6.50
3 Chiefs vs. Waratahs Jun 03 Chiefs 12.70
4 Brumbies vs. Rebels Jun 03 Brumbies 21.60
5 Force vs. Hurricanes Jun 03 Hurricanes -22.90
6 Jaguares vs. Kings Jun 30 Jaguares 11.10
7 Lions vs. Sunwolves Jul 01 Lions 32.30
8 Cheetahs vs. Stormers Jul 01 Stormers -6.90
9 Sharks vs. Bulls Jul 01 Sharks 10.20

 

NRL Predictions for Round 13

Team Ratings for Round 13

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Current Rating Rating at Season Start Difference
Storm 7.07 8.49 -1.40
Broncos 5.69 4.36 1.30
Sharks 5.18 5.84 -0.70
Raiders 5.11 9.94 -4.80
Sea Eagles 3.25 -2.98 6.20
Roosters 2.88 -1.17 4.00
Cowboys 1.65 6.90 -5.30
Dragons 1.48 -7.74 9.20
Panthers 0.45 6.08 -5.60
Titans -1.68 -0.98 -0.70
Eels -1.80 -0.81 -1.00
Bulldogs -2.12 -1.34 -0.80
Warriors -3.50 -6.02 2.50
Rabbitohs -4.69 -1.82 -2.90
Wests Tigers -8.31 -3.89 -4.40
Knights -12.69 -16.94 4.20

 

Performance So Far

So far there have been 92 matches played, 52 of which were correctly predicted, a success rate of 56.5%.
Here are the predictions for last week’s games.

Game Date Score Prediction Correct
1 Rabbitohs vs. Eels May 26 16 – 22 1.90 FALSE
2 Warriors vs. Broncos May 27 28 – 10 -9.40 FALSE
3 Sharks vs. Bulldogs May 27 9 – 8 12.60 TRUE
4 Raiders vs. Roosters May 28 24 – 16 5.20 TRUE

 

Predictions for Round 13

Here are the predictions for Round 13. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Game Date Winner Prediction
1 Storm vs. Knights Jun 02 Storm 23.30
2 Eels vs. Warriors Jun 02 Eels 5.70
3 Dragons vs. Wests Tigers Jun 03 Dragons 13.30
4 Roosters vs. Broncos Jun 03 Roosters 0.70
5 Cowboys vs. Titans Jun 03 Cowboys 6.80
6 Sea Eagles vs. Raiders Jun 04 Sea Eagles 1.60
7 Bulldogs vs. Panthers Jun 04 Bulldogs 0.90

 

May 29, 2017

The past is a foreign country

As you might have read in an English class (at least if you’re old) “The past is a foreign country: they do things differently there.” In particular, they buy things differently.

We often want to compare prices between the present and the past. To compare prices between here and actual foreign countries, we use the exchange rate and find, for example, that the US cost (from Amazon) of Sennheiser CX 3.0 earphones, US$29.99 is equivalent to about NZ$40, or about NZ$46 with tax.  We can then straightforwardly say that Harvey Norman is charging 60% more  than Amazon for the same item, and force them to make unconvincing excuses.  Comparing foreign prices can get complicated if you’re interested in affordability relative to income, or if you’re looking at a country with very restrictive border controls but the existence of two-way trade with many foreign countries means there is a single, well-defined exchange rate.

The equivalent conversion for past prices is an inflation adjustment. If you’re comparing past and current prices without an inflation adjustment you’re not even trying to get the numbers right (with a few very limited exceptions such as bracket creep).  There should be an automatic presumption of dodginess for any `nominal’, unadjusted comparison of amounts of money at different times — especially as this is something that’s really easy to get (approximately) right.  So, go and fix it now.

I said “approximately right”, and those of you still with me will note that we don’t have extensive two-way trade with the past. Inflation isn’t as simple as an exchange rate.You can’t buy Sennheiser CX 3.0 earphones with 1997 dollars, and even if you could, Amazon would have difficulty shipping them to 1997.  Inflation adjustments are more like the Economist‘s Big Mac purchasing-power index.  The magazine decrees that Big Macs have the same true value everywhere in the world, and so can estimate the relative value of different currencies.  That’s not ideal when comparing countries, and it’s even harder when comparing with the past.

Economists and official statistics agencies make up ‘baskets’ of items and decree these to have the same value over time, carefully making sure that the items are defined narrowly enough for ‘the same value’ to be reasonable, and broadly enough that you can still find ‘the same item’ a year later.  They also make complicated adjustments for changes in quality of the ‘same item’ (math is hard even if you go shopping).  That’s really the only way you can do it, and it works ok for many purposes, but there isn’t just one inflation rate the way there is one exchange rate.

Macroeconomists use this thing called ‘core inflation’, which leaves out items that vary a lot in price and which they say predicts macroeconomic things better.  There are indexes based on baskets of items bought by NZ producers, or sold by NZ producers.  There are indexes based on baskets of items relevant to different sorts of households: Graeme Edgeler has a nice post pointing out that there is an inflation index targeted to beneficiaries, and that it would make sense to use this to index benefits (he also drafted a bill).

The other tricky part of the basket approach to comparing past and current prices is the huge differences between items.  The prices of computers, washing machines, t-shirts, and old-enough medications are increasing more slowly than the average inflation rate; they are getting systematically less expensive ‘in real terms’. By simple arithmetic, that means the prices of other things must be rising faster than the average inflation rate; they must be getting more expensive ‘in real terms’.

It sounds odd to say that primary schools or health care are getting more expensive to run `in real terms’ because computers and t-shirts are cheaper — but it’s partly true. Some sort of currency conversion is always necessary when comparing different currencies — the 2001 dollar and the 2017 dollar — but it’s not always sufficient.

 

Stat of the Week Competition: May 27 – June 2 2017

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

  • Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday June 2 2017.
  • Statistics can be bad, exemplary or fascinating.
  • The statistic must be in the NZ media during the period of May 27 – June 2 2017 inclusive.
  • Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

May 26, 2017

Big fat lies?

This is a graph from the OECD, of obesity prevalence:
oecd

The basic numbers aren’t novel. What’s interesting (as @cjsnowdon pointed out on Twitter) is the colour separation. The countries using self-reported height and weight data report lower rates of obesity than those using actual measurements.  It wouldn’t be surprising that people’s self-reported weight, over the telephone, tends to be a bit lower than what you’d actually measure if they were standing in front of you; this is a familiar problem with survey data, and usually we have no real way to tell how big the bias is.

In this example there’s something we can do.  The United States data come from the National Health And Nutrition Examination Surveys (NHANES), which involve physical and medical exams of about 5,000 people per year. The US also runs the Behavioral Risk Factor Surveillance System (BRFSS), which is a telephone interview of half a million people each year. BRFSS is designed to get reliable estimates for states or even individual counties, but we can still look at the aggregate data.

Doing the comparisons would take a bit of effort, except that one of my students, Daniel Choe, has already done it. He was looking at ways to combine the two surveys to get more accurate data than you’d get from either one separately.  One of his graphs shows a comparison of the obesity rate over a 16-year period using five different statistical models. The top right one, labelled ‘Saturated’, is the raw data.
combined

In the US in that year the prevalence of obesity based on self-reported height and weight was under 30%.  The prevalence based on measured height and weight was about 36% — there’s a bias of about 8 percentage points. That’s nowhere near enough to explain the difference between, say, the US and France, but it is enough that it could distort the rankings noticeably.

As you’d expect, the bias isn’t constant: for example, other research has found the relationship between higher education and lower obesity to be weaker when using real measurements than when using telephone data.  This sort of thing is one reason doctors and medical researchers are interested in cellphone apps and gadgets such as Fitbit — to get accurate answers even from the other end of a telephone or internet connection.