Stats Chat

The Herald has an annoyingly uncritical story about someone who claims to have a mathematical formula for winning the lottery, rather than just being lucky.

Much more interesting: BBC’s More or Less had a story about multiple lottery wins and how they might come about.

• Auckland Transport commissioned market research firm TRA to do a survey on attitudes to bike paths. Mike Hosking didn’t like the results. Hilarity ensues.
• Mike Bostock looks at ways of comparing data on maps
• “Household Water Use During Argentina-Iceland Game Looks Like You’d Expect” from Reykjavik Grapevine (who are clearly enjoying the World Cup)
• mathwashing.com is a website about, unsurprisingly, ‘mathwashing’ — making a bad decision look better by saying it was made by a computer (or an algorithm).  It’s completely one-sided, but it’s still a useful corrective.
• Pew Research tried to study the extent to which people distinguish between assertions of fact and statements of opinion in the news. It’s harder than it sounds.
• Janelle Shane writes about how you can distinguish pastiche written by a neural network from parody written by people

Team Ratings for Round 16

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 116 matches played, 68 of which were correctly predicted, a success rate of 58.6%.
Here are the predictions for last week’s games.

Predictions for Round 16

Here are the predictions for Round 16. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Journalists are often reluctant to attribute causes for individual events — the journalistic use of ‘after’ and ‘amid’ leads readers to conclusions in a much more deniable way — but less reluctant for groups.

Today, there’s an interesting range of descriptions of some research

Herald: Automobile Association study finds drugs cause more fatal crashes than alcohol

Stuff: Drug-impaired drivers now involved in more fatal crashes than drink-drivers

Radio NZ: Drugged driving fatalities outnumber drink driving deaths

NZ Autocar: AA FINDS MARKED INCREASE IN DRUG DRIVING FATALITIES

None of these link to the actual study report, and when I first looked, the report was on the AA website or on Scoop, but it the results table is in the NZ Autocar story. (Scoop has a press release from 10:49am; it’s still not on the AA website, where the most recent media release is dated 21 May)

None of the headlines is supported all that well by the data.

The data are based on what’s recorded in the ‘Crash Analysis System’, and is based on blood alcohol above the legal level and on presence of illegal drugs or prescription drugs that might have impaired the driver.  It’s not based on actual impairment (which is obviously hard to measure after the crash). The Crash Analysis System is set up to record anything that might be a contributing cause, because if some factor doesn’t make it into the database there’s no way to go back and check it later. Back in 2015 when the data were public and I looked at them, the system averaged about 2 1/4 causes per crash.

We pretty much know that most crashes where the driver is not far above the legal blood alcohol limit are not caused by alcohol — that’s the whole point of setting the threshold where it is.  For some illegal drugs — notably, cannabis — there isn’t a good test for impairment in regular users.

The AA says that other countries are using roadside testing. They are, and that’s partly because some countries regard the false positives  — catching someone who has used illegal drugs but isn’t impaired at the time — as a feature, not a bug.  The combination of alcohol and cannabis does seem to be a real problem, and US expert Mark Kleiman has suggested a blood alcohol threshold of zero for people who use cannabis.

But on top of that, as the NZ Autocar story says

While the numbers suggest drug driving has suddenly skyrocketed, the AA believes the big jump is likely down to more thorough testing being done following crashes.

That’s also in the press release. But it’s not in the other stories.

Team Ratings for Round 15

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 108 matches played, 65 of which were correctly predicted, a success rate of 60.2%.
Here are the predictions for last week’s games.

Predictions for Round 15

Here are the predictions for Round 15. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

• “Greece’s supreme court has rejected an appeal by the country’s former statistics chief to quash his conviction in 2017 on charges of violation of duty” (Financial Times, previous comment by American Statistical Association and Royal Statistical Society)
• The government of Puerto Rico has finally released official mortality statistics for the time since the hurricanes. Rafa Irizarry writes about them at Simply Statistics.
• The (UK) Academy of Medical Sciences has recommended a ‘nutrition label’ for medical science press releases, with indicators for peer-reviewed or not, whether it was in people or not, and what type of study if it was in people. And, according to the (UK) Science Media Centre, there was much rejoicing.
• Showing the distribution of income across different job categories, rather than just mean or median: Flowing Data
• Animation from NIWA of the NZ tides.

Last week, Microsoft (who you’ve probably heard of) bought GitHub (who you may well not have heard of) with $7.5 billion in Microsoft stock.  Github is a site that cross-bred version-control software (‘track changes’ for programmers) with social media, providing a place to share and promote code.

Stuff and the Herald quoted the same number

More than 28 million developers around the world use GitHub, with Microsoft ranking as the most active organisation on GitHub.

You might wonder what proportion of developers use GitHub. If you search a bit on Google, you’ll find that the total number of developers is estimated at about 21 million, so roughly 130% of them use GitHub.

Obviously there’s something wrong there.  The problem is how to define ‘developers’.

In the US, the Bureau of Labor Statistics reports how many people do each job. They say, currently,  that there are 1,617,400 “Software Developers and Programmers”.  The figure of 28 million worldwide is actually based on a subset of those, the 1.2 million “Software Developers, Applications” and “Software Developers, Systems”.  This sort of official classification has to be narrow, because the goal is for  every job to end up in exactly one category.

Lots of people in the US who are developers in the GitHub sense aren’t developers in the Bureau of Labor Statistics sense. Some of them write software only in their spare time. Others write software as part of their jobs, but their jobs are classified somewhere else in the BLS system. The same is true in New Zealand. Stats NZ reports 31,860 jobs in ANZSIC06 category 7000 “Computer Systems Design and Related Services”, which is a bit broader than the US category.  Even though I’m a developer in the GitHub sense, I’m not one of them. I’m in 8102, “Higher Education”.

Other people who probably in the 21 million count include statisticians, data scientists, computational biologists, ornithologists, journalists, linguists, and many more.

Official statistics are usually pretty accurate, but they are only accurate for what they are trying to measure, which might not be what you are looking for.

The new QS university rankings are out, and there’s a story in the Herald.  It starts

Staff cuts despite growing student numbers have dragged most New Zealand universities down in the latest world rankings.

The biggest six of the country’s eight universities have all tumbled in the London-based QS rankings, which are regarded as the most important for attracting international students.

“Tumbled” is an exaggeration — for example, the University of Auckland has ‘tumbled’ from 82nd to 85th in the rankings. But the message that it’s staff numbers does seem to be backed up with a quote from QS

“The increase in enrolments – and the decrease in faculty numbers – reported by the country’s universities sees all eight receive a lower score for faculty/student ratio,”

QS don’t make it easy to find older numbers, but an archive of their webpage in March last year said there were 29,930 students and 2025 academic staff, and a ranking of 81. The current figures are  29,641  students (ie, fewer) and 2,047 academic staff (ie, more), for an improvement in staff:student ratio from 14.8 to 14.5. 

That’s over a two year period, but last year, the story at Stuff said

New Zealand universities performed well in research outputs – Waikato ranked 133rd, Otago 174th and Canterbury 178th – but showed “uniformly deteriorating” faculty to student ratios. The exception was Lincoln University, which featured among the top 200 universities globally in that measure.

So, cumulatively over this two year period the staff:student ratio at UoA (as measured by QS) improved, but the reporting said it worsened in both years.

My guess as to what’s going on is that these rankings are rankings. What they mean by “lower score for faculty/student ratio” isn’t that the ratio got worse here, but that it got better here by less than it did at some competing universities.

The other strange thing in the Herald story is this:

The worsening staff/student ratio in NZ universities was entirely due to cuts of 203 academics at Massey and 74 at Lincoln.

It could be true that these are the only NZ universities with a worsening staff/student ratio — the other universities could be seeing the same sort of change that UoA did — but if it is, the apparent contradiction with the lead should have been noticed.

Team Ratings for Round 17

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 99 matches played, 69 of which were correctly predicted, a success rate of 69.7%.
Here are the predictions for last week’s games.

Predictions for Round 17

Here are the predictions for Round 17. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

• • According to the Daily Mail “Brisk ten-minute walk, around 1,000 steps, can increase mortality by 15 per cent.”  They mean decrease.
  • Wikipedia thinks Amanda Cox, the data visualisation expert from the New York Times, isn’t “notable”. They are wrong and they should feel bad. But also, published evidence that she is notable would be useful.
  • The FDA has issued guidance on how to include pregnant women in clinical trials.
  • A monthly collection of the best data visualisation from around the web
  • Black cats are less popular

Debuting new survival analysis example in class today (from Austin TX’s animal control data available online). pic.twitter.com/9qDmFMKvbM

— Jeremy Freese (@jeremyfreese) May 30, 2018