Stats Chat

Online polling company YouGov asked people in the US and Britain about how their intelligence compared to other people.

For the US, the results were

They pulled that graph only seconds after I found it, and replaced it with the more plausible

The British appear to be slightly more reluctant that the Americans to say they’re smarter than average, though it would be unwise to assume they are less likely to believe it.

Keith Ng has his annual interactive graphic of budget changes up at Public Address, and will soon have a graphic showing how overall forecasts have changed over time.

[update] And Harkanwal Singh has his version up at the Herald

There’s a Stat-of-the-Week nomination for a Dominion Post article that I haven’t seen, because Stuff has had the good sense not to put it online. The press release is on Scoop, and from what our correspondent says, if you’ve read that, you’ve read the story. It’s about sex at the office, based on a ridiculously small sample selected from members of a dating website.  Since the dating website in question makes a lot of how different its members are from typical people, representativeness is not likely. Also, their infographic disagrees with the text of the release in at least one place.

That’s all standard. What’s interesting is the comparison of proportion of men and women who have had sex in various situations. Now, for the heterosexual majority, we have a basic accounting constraint in play. The office-sex survey says 20% of men and 3% of women have got it on in a conference room and 15% of men and 2% of women have done so in a storage room.   If these numbers were true there would be only three explanations: there are a lot more gay men around than other data suggest, and they really like the office; the few women who have sex at the office do so with many different men; or we have a Clintonesque definitional problem where the vast majority of the women involved don’t think what they did was sex.  More likely, it’s just evidence that the numbers are meaningless.

We’ve seen this problem before, but at least this is one problem the Herald’s story about holiday romance based on an Expedia press release avoided.

[Update: 538 now has an intro to the story explaining the mistakes and apologising. Good for them.]

So, at  fivethirtyeight.com there’s this story on mapping kidnappings in Nigeria using data from GDELT, the sort of thing data journalism is supposed to be good at. GDELT automatically extracts information from news stories to build a huge global database.

On Twitter, Erin Simpson, whose about.me page says she is “a leading specialist in the intersection of intelligence, data analysis, irregular warfare, and illicit systems – with an emphasis on novel research designs,” — and who has worked on the GDELT parser — is Not Happy.

Thanks to Storify, here are three summaries of what she says, but a lot of it can be boiled down to one point:

In conclusion: VALIDATE YOUR FREAKING DATA. It’s not true just because it’s on a goddamn map.

(via @LewSOS)

Team Ratings for Round 10

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 72 matches played, 39 of which were correctly predicted, a success rate of 54.2%.

Here are the predictions for last week’s games.

Predictions for Round 10

Here are the predictions for Round 10. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 14

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 81 matches played, 54 of which were correctly predicted, a success rate of 66.7%.

Here are the predictions for last week’s games.

Predictions for Round 14

Here are the predictions for Round 14. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

From the Herald

Housing Minister Nick Smith has revealed that Government held data on the proportion of New Zealand homes owned by offshore buyers, which he says is very low compared to other countries.

It turns out that the Government actually has data on the proportion of rental landlords who are overseas. Not the proportion of all homes, and not even the proportion of all rental homes. And even then, the proportion is based on whether the landlords are currently offshore, not whether they were offshore at the time of purchase, which is the topic of controversy (as the Herald does note).

It’s hard to do anything about the landlords vs rentals difference, but if the proportion of rentals owned offshore was also 11%, that would translate to about 4% of New Zealand homes, based on the home ownership figures from the Census.

Two new(ish) interactive visualisations

• From Harkanwal Singh, in the Herald, a map of the new edition of the NZ deprivation index
• From Stats New Zealand, the national progress indicators

Q: Did you see “Cocaine now on tap in British homes” in the Herald

A: Yes.

Q: Is it true?

A: Not so as you’d notice.

Q: Didn’t they find traces of cocaine in drinking water?

A: Up to a point.

Q: You mean no?

A: I mean they found traces of the chemical that cocaine gets broken down into

Q: And is that a drug?

A: Not really. It was one component of an unsuccessful treatment for back pain. It is restricted, because it can be turned into cocaine.

Q: How much of this stuff did they find?

A: Almost none. A few nanograms per litre

Q: What’s that in real numbers? If it was really cocaine, how long would it take you to get one dose if you drank  eight glasses of water a day like the doctors recommend?

A: That isn’t actually what the doctors recommend.

Q: Well, then, “like the doctors don’t recommend?”

A: Several centuries.

Q: How can they detect such tiny amounts?

A: They use liquid chromatography to separate out each chemical, and then mass spectroscopy to basically count the molecules.

Q: Ok, impressed now.  The story also mentions “significant amounts of caffeine”. What does that mean?

A: It means “insignificant amounts”, about a million times lower concentration than in a cup of decaffeinated coffee.

Q: At least this is new, though?

A: The same agency reported finding the cocaine metabolite in drinking water in 2011, based on measurements in 2009-10. (PDF, Table 6)

Q: Why is there a video of a drug bust in Spain embedded in the story?

A: Because technology.

Attention conservation notice: I have to write this post because I’ve spent too much time on it otherwise. You don’t have to read it.

There was an episode of “Yes, Prime Minister” where the term “Human Resource Rich Countries” was being posed as a replacement for “Less Developed Countries”, meaning “poor”. “Resources” is a word that can mean lots of different things, which is why I spent more time than was strictly sensible investigating the following graph

The graph appeared in my Twitter feed last Monday. It’s originally from a campaign to give Australia a school funding model a bit more like NZ’s decile system, as recommended by a national review panel, so it is disturbing to see New Zealand almost at the bottom of the world.

(more…)