Stats Chat

This graph was on Twitter this morning. There’s nothing wrong with the graph: good data, clear presentation, but it does provide a nice illustration of the difficulties in official statistics — you have to decide what categories to use, and it makes a difference.

The second leading cause, motor vehicles, is straightforward enough.  The first, firearms, is more complicated. A majority of the firearm deaths are suicides, and it’s controversial whether firearm access increases the suicide rate or just affects the method.  Poisoning is also complicated: you might well want to treat both suicide and accidental recreational-drug overdose separately. And so on.

Sometimes you want to break down the data by intent, sometimes by physical cause, sometimes by medical type of injury or damage. You can’t define the ‘correct’ answer in the absence of a question.

The second annual edition of the Ihaka Lecture Series has just ended, and we are, once again, delighted with the turnout and engagement, in person and online. Our final speaker was Alberto Cairo, right, Knight Chair in Visual Journalism at the University of Miami, whose lecture on the dubious uses of data was thought-provoking and a bit worrying.

If you want to see how Trump supporters deluded themselves and misled others with graphics, it’s all laid bare here in Alberto’s lecture. And that this brand of Trumpery is not the only example of statistics willfully used to mislead – Alberto delivers a few other eye-openers. And some laughs, as well – he is a very entertaining and engaging speaker. By the way, it’s not all bad news – there is much useful and thoughtful work being done, and Alberto shows what that is and where.

Alberto’s lecture is accessible to all. He uses non-technical language, as and Alberto says, he’s not a statistician. So if you are teaching secondary-school statistics (or citizenship or social studies … ) this would be a really good resource for your students.

Also, Alberto was yesterday interviewed by Colin Peacock, the long-time host of Radio New Zealand’s Mediawatch, and it’s recommended listening. The pic Mediawatch ran of Alberto on its webpage was so nice, we stole it. Nice image, RNZ’s Claire Eastham-Farrelly!

Of course, we also had two other incomparable speakers: our own Associate Professor Paul Murrell, one of the movers and shakers behind R, on the BrailleR package, which generates text descriptions of R plots (watch here) and Monash Professor Dianne Cook, who described some simple tools for helping to decide if patterns you think you are seeing in the data are really there (watch here).

And … in breaking news, the theme of next year’s Ihaka Lecture Series is … machine learning! Speakers will be announced at a later date.

+ Useful link: The 2017 Ihaka Lecture Series.

Team Ratings for Round 6

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 30 matches played, 20 of which were correctly predicted, a success rate of 66.7%.
Here are the predictions for last week’s games.

Predictions for Round 6

Here are the predictions for Round 6. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 3

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 16 matches played, 8 of which were correctly predicted, a success rate of 50%.
Here are the predictions for last week’s games.

Predictions for Round 3

Here are the predictions for Round 3. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

• “Bus Lane Blocked, He Trained His Computer to Catch Scofflaws” from the New York Times. It would be hard for this to be more relevant to my interests.
• NZ ‘citizen science’ analyses suggesting the larger-battery versions of the Nissan Leaf have battery life problems. Peter Griffin at Noted.co.nz
• Wikipedia has the best data on US historical city/town populations, thanks to the work of one person who has entered the data by hand. Here’s a visualisation of it.
• A bit more technical than usual: how to understand what a neural network is doing when it classifies images (from distill.pub)
• From Source, “How We Found New Patterns in LA’s Homeless Arrest Data”

This is starting to look like a series. The Herald today has Warning: Doomsday asteroid taller than Empire State building cannot be stopped by Nasa. They go on to say

The consequences would be “dire” experts have warned, and the asteroid has sparked fears that it could even wipe out life for good.

Stuff has the less-exciting headline Nasa draws up plans for huge eight-ton spacecraft to blow up doomsday asteroid. You might be able to guess which one is more accurate.

If you follow the link from the Herald to the Daily Mail, and then from the Daily Mail to the scientific research paper, and then get past the paywall, you find the research is about the best ways to handle an asteroid like this one if it turns out to be a threat. As the Stuff story says, the scientists think they could manage a deflection given enough notice; with less warning than that they might need to blow it up with a big nuke.

So, how much warning do we have? The Herald says

Based on observational data, Bennu has a 1 in 2,700-chance of striking Earth on Sept. 25, 2135.

That’s quite a long time in the future — if scientists think they’ve got a good chance of deflecting it even with today’s technology, we should be ok even without Bruce Willis.

That’s if we believe the numbers in the story. Let’s check them. The research paper points to a lovely website from NASA that lists all the object we know about that could get scarily close to Earth at some point in the future.  Here’s their entry for Doomsday Asteroid 101955 Bennu.

Bennu has a cumulative impact probability of 3.7×10^-4, ie, 1/2,700.  That’s added up over the entire foreseeable future. There are 78 potential impact dates listed. The first is 25th September 2175, with a probability of 4.1×10^-5, or about 1/25,000.  The remainder of the 1/2700 probability is spread over the following 25 years.  So, the date is wrong, and the probability is misleading. Since it looks like the Daily Mail got the numbers from Buzzfeed, that’s a bit disappointing.

The NASA site also lists 101955 Bennu as -1.71 on the “Palermo scale.” The Palermo scale compares the risk of being hit by a specific asteroid over a period of time to the expected risk of being hit by all the other asteroids of the same size that we don’t know about yet. Bennu’s -1.71 means the risk from Bennu is 10^-1.71 times lower than the background risk — about 50 times lower. Dealling with Bennu would lower our risk over the next couple of centuries for asteroids of that size by a few percent. It’s worth doing — but mostly as a test of the technology.

Also, it would be interesting to know who is actually afraid the asteroid “could wipe out life for good”. It’s presumably not the “experts” from earlier in the sentence: the impact is estimated as having the energy of a 1.15 gigaton explosion.  That’s not something you want in your school zone, but it’s tiny compared to, say, the Taupo eruption roughly 2000 years ago. It’s about of the order of magnitude of the Tarawera explosion in 1886.  An impact in the worst possible place — maybe on top of Shanghai —  could kill a lot of people even with the best evacuation efforts but it wouldn’t wipe out industrial civilisation, let alone all life.

(hat tip: Mark Hanna)

There’s a new paper in the journal Nature Human Behaviour analysing a huge collection of election poll data: over 30,000 polls. The researchers’ conclusion is straightforward: polls have not become less accurate. Unfortunately, all the nice graphs are behind a paywall. Fortunately, the data isn’t, and I can draw you a nice graph of my own

The graph shows all the poll results back to 1970, split up into panels by how many weeks before the election they were.  I’m showing just one party per poll: the data have conveniently been coded so it’s a big party (eg Labour for NZ, Conservatives for UK). Each panel shows the error in the poll plotted against the year of the election; the red line is an average.

The red lines are basically flat. Despite cellphones, the internet, political polarisation, and millennials, average polling error hasn’t changed all that much over the past fifty years.

Team Ratings for Round 5

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 23 matches played, 15 of which were correctly predicted, a success rate of 65.2%.
Here are the predictions for last week’s games.

Predictions for Round 5

Here are the predictions for Round 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 2

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 8 matches played, 3 of which were correctly predicted, a success rate of 37.5%.
Here are the predictions for last week’s games.

Predictions for Round 2

Here are the predictions for Round 2. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Astronaut Scott Kelly has been extensively studied after a year in space (and so has his identical twin).  There’s a new, and pretty dramatic, story about some of the results. For example, IFLScience says NASA Sent One Identical Twin Brother To Space For A Year – And It May Have Permanently Changed 7 Percent Of His DNA. So does Business Insider.

If you know that a chimpanzee’s DNA is only about 1% different from a human’s — or that a mouse’s is about 8% different — that sounds weird. It’s even worse than that: the chance you’d still be alive after that sort of mutation load is pretty small.

So what did happen? Well, the story seems to be an example of accumulated mutations itself. In a recent interview for Marketplace, Scott Kelly said

“I did read in the newspaper the other day… that 7 percent of my DNA had changed permanently,” Kelly said. “And I’m reading that, I’m like, ‘Huh, well that’s weird.’” 

We’re seeing reports of someone quoting something from the media, rather than any primary source. If you go to the NASA press release, it says

Although 93% of genes’ expression returned to normal postflight, a subset of several hundred “space genes” were still disrupted after return to Earth.

That seems to be the origin of the ‘7%’ figure.

So what’s the difference? Imagine the genome as a library.  A 7% chance in DNA would be like saying 7% of the words in all the books in the library had been altered.  A change in expression in 7% of genes would be like 7% of the books having a noticeable increase or decrease in how often they were borrowed.

There were also some small changes in Scott’s DNA.  His telomeres, which are the caps on chromosomes that stop them fraying at the ends (like the little plastic bits on shoelaces) were slightly longer — which is probably good. DNA that scientists sequenced from his blood also had “hundreds” of new mutations: more than you’d typically expect, but still only about 0.0000001% of his DNA