Stats Chat

Team Ratings for Round 3

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 16 matches played, 8 of which were correctly predicted, a success rate of 50%.
Here are the predictions for last week’s games.

Predictions for Round 3

Here are the predictions for Round 3. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

• “Bus Lane Blocked, He Trained His Computer to Catch Scofflaws” from the New York Times. It would be hard for this to be more relevant to my interests.
• NZ ‘citizen science’ analyses suggesting the larger-battery versions of the Nissan Leaf have battery life problems. Peter Griffin at Noted.co.nz
• Wikipedia has the best data on US historical city/town populations, thanks to the work of one person who has entered the data by hand. Here’s a visualisation of it.
• A bit more technical than usual: how to understand what a neural network is doing when it classifies images (from distill.pub)
• From Source, “How We Found New Patterns in LA’s Homeless Arrest Data”

This is starting to look like a series. The Herald today has Warning: Doomsday asteroid taller than Empire State building cannot be stopped by Nasa. They go on to say

The consequences would be “dire” experts have warned, and the asteroid has sparked fears that it could even wipe out life for good.

Stuff has the less-exciting headline Nasa draws up plans for huge eight-ton spacecraft to blow up doomsday asteroid. You might be able to guess which one is more accurate.

If you follow the link from the Herald to the Daily Mail, and then from the Daily Mail to the scientific research paper, and then get past the paywall, you find the research is about the best ways to handle an asteroid like this one if it turns out to be a threat. As the Stuff story says, the scientists think they could manage a deflection given enough notice; with less warning than that they might need to blow it up with a big nuke.

So, how much warning do we have? The Herald says

Based on observational data, Bennu has a 1 in 2,700-chance of striking Earth on Sept. 25, 2135.

That’s quite a long time in the future — if scientists think they’ve got a good chance of deflecting it even with today’s technology, we should be ok even without Bruce Willis.

That’s if we believe the numbers in the story. Let’s check them. The research paper points to a lovely website from NASA that lists all the object we know about that could get scarily close to Earth at some point in the future.  Here’s their entry for Doomsday Asteroid 101955 Bennu.

Bennu has a cumulative impact probability of 3.7×10^-4, ie, 1/2,700.  That’s added up over the entire foreseeable future. There are 78 potential impact dates listed. The first is 25th September 2175, with a probability of 4.1×10^-5, or about 1/25,000.  The remainder of the 1/2700 probability is spread over the following 25 years.  So, the date is wrong, and the probability is misleading. Since it looks like the Daily Mail got the numbers from Buzzfeed, that’s a bit disappointing.

The NASA site also lists 101955 Bennu as -1.71 on the “Palermo scale.” The Palermo scale compares the risk of being hit by a specific asteroid over a period of time to the expected risk of being hit by all the other asteroids of the same size that we don’t know about yet. Bennu’s -1.71 means the risk from Bennu is 10^-1.71 times lower than the background risk — about 50 times lower. Dealling with Bennu would lower our risk over the next couple of centuries for asteroids of that size by a few percent. It’s worth doing — but mostly as a test of the technology.

Also, it would be interesting to know who is actually afraid the asteroid “could wipe out life for good”. It’s presumably not the “experts” from earlier in the sentence: the impact is estimated as having the energy of a 1.15 gigaton explosion.  That’s not something you want in your school zone, but it’s tiny compared to, say, the Taupo eruption roughly 2000 years ago. It’s about of the order of magnitude of the Tarawera explosion in 1886.  An impact in the worst possible place — maybe on top of Shanghai —  could kill a lot of people even with the best evacuation efforts but it wouldn’t wipe out industrial civilisation, let alone all life.

(hat tip: Mark Hanna)

There’s a new paper in the journal Nature Human Behaviour analysing a huge collection of election poll data: over 30,000 polls. The researchers’ conclusion is straightforward: polls have not become less accurate. Unfortunately, all the nice graphs are behind a paywall. Fortunately, the data isn’t, and I can draw you a nice graph of my own

The graph shows all the poll results back to 1970, split up into panels by how many weeks before the election they were.  I’m showing just one party per poll: the data have conveniently been coded so it’s a big party (eg Labour for NZ, Conservatives for UK). Each panel shows the error in the poll plotted against the year of the election; the red line is an average.

The red lines are basically flat. Despite cellphones, the internet, political polarisation, and millennials, average polling error hasn’t changed all that much over the past fifty years.

Team Ratings for Round 5

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 23 matches played, 15 of which were correctly predicted, a success rate of 65.2%.
Here are the predictions for last week’s games.

Predictions for Round 5

Here are the predictions for Round 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 2

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 8 matches played, 3 of which were correctly predicted, a success rate of 37.5%.
Here are the predictions for last week’s games.

Predictions for Round 2

Here are the predictions for Round 2. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Astronaut Scott Kelly has been extensively studied after a year in space (and so has his identical twin).  There’s a new, and pretty dramatic, story about some of the results. For example, IFLScience says NASA Sent One Identical Twin Brother To Space For A Year – And It May Have Permanently Changed 7 Percent Of His DNA. So does Business Insider.

If you know that a chimpanzee’s DNA is only about 1% different from a human’s — or that a mouse’s is about 8% different — that sounds weird. It’s even worse than that: the chance you’d still be alive after that sort of mutation load is pretty small.

So what did happen? Well, the story seems to be an example of accumulated mutations itself. In a recent interview for Marketplace, Scott Kelly said

“I did read in the newspaper the other day… that 7 percent of my DNA had changed permanently,” Kelly said. “And I’m reading that, I’m like, ‘Huh, well that’s weird.’” 

We’re seeing reports of someone quoting something from the media, rather than any primary source. If you go to the NASA press release, it says

Although 93% of genes’ expression returned to normal postflight, a subset of several hundred “space genes” were still disrupted after return to Earth.

That seems to be the origin of the ‘7%’ figure.

So what’s the difference? Imagine the genome as a library.  A 7% chance in DNA would be like saying 7% of the words in all the books in the library had been altered.  A change in expression in 7% of genes would be like 7% of the books having a noticeable increase or decrease in how often they were borrowed.

There were also some small changes in Scott’s DNA.  His telomeres, which are the caps on chromosomes that stop them fraying at the ends (like the little plastic bits on shoelaces) were slightly longer — which is probably good. DNA that scientists sequenced from his blood also had “hundreds” of new mutations: more than you’d typically expect, but still only about 0.0000001% of his DNA

Jamie Morton has an interesting story in the Herald, reporting on research by Wellington firm Dot Loves Data.

They then investigated how well they all predicted the occurrence of assaults at “peak” times – between 10pm and 3am on weekends – and otherwise in “off-peak” times.

Unsurprisingly, a disproportionate number of assaults happened during peak times – but also within a very short distance of taverns.

The figures showed a much higher proportion of assault occurred in more deprived areas – and that, in off-peak times, socio-economic status proved a better predictor of assault than the nearness or number of bars.

Unsuprisingly, the police were unsurprised.

This isn’t just correlation: with good-quality location data and the difference between peak and other times, it’s not just a coincidence that the assaults happened near bars, nor is it just due to population density.  The closeness of the bars and the assaults also argues against the simple reverse-causation explanation: that bars are just sited near their customers, and it’s the customers who are the problem.

So, it looks as if you can predict violent crimes from the location of bars (which would be more useful if you couldn’t just cut out the middleman and predict violent crimes from the locations of violent crimes).  And if we moved the bars, the assaults would probably move with them: if we switched a florist’s shop and a bar, the assaults wouldn’t keep happening outside the florist’s.

What this doesn’t tell us directly is what would happen if we dramatically reduced the number of bars.  It might be that we’d reduce violent crime. Or it might be that it would concentrate around the smaller number of bars. Or it might be that the relationship between bars and fights would weaken: people might get drunk and have fights in a wider range of convenient locations.

It’s hard to predict the impact of changes in regulation that are intended to have large effects on human behaviour — which is why it’s important to evaluate the impact of new rules, and ideally to have some automatic way of removing them if they didn’t do what they were supposed to.  Like the ban on pseudoephedrine in cold medicine.

A bit more technical than usual, but definitely worth reading: “Reflections on Quantitative Fairness”

A couple of less-technical excerpts

Much communication consists of taking one or another of these fairness concepts as obvious or axiomatic and asserting the violation of that principle as a political or moral gotcha. Formalization should not be regarded as a panacea in these debates but perhaps it can help to cement the points that:

• a lack of clarity can conceal a debate with real content and stakes
• differences in priorities and understandings of fairness are actually unresolved and in principle unresolvable without trade-offs

and

As statistical thinkers in the political sphere we should be aware of the hazards of supplanting politics by an expert discourse. In general, every statistical intervention to a conversation tends to raise the technical bar of entry, until it is reduced to a conversation between technical experts. As a result, in matters of criminal justice, public health, and employment, the key stakeholders, whose stakes are human stakes, and who typically lack a statistical background, can easily fall out of the conversation.

So are we speaking statistics to power? Or are we merely providing that power with new tools for the marginalization of unquantified political concerns? What is the value of this quantitative fairness conversation to a person or community whose concerns will not be quantified for another decade, if ever?

That is: it’s worth trying to be clear about what the actual question is, but we have to be careful in doing that not to push out the people who know the answer.

Team Ratings for Round 4

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 16 matches played, 11 of which were correctly predicted, a success rate of 68.8%.
Here are the predictions for last week’s games.

Predictions for Round 4

Here are the predictions for Round 4. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.