Stats Chat

• For the day between March 31 and April 2nd, Andrew Gelman takes on an app that claims to find patterns in lotto numbers and make you money.
• RNZ reports the plans for tolls on the Road of Northland Significance, a charge of $4.50 each way from Warkworth to Te Hana (you will see some quotes of $14.20, which includes current tolls on the already-existing road to Puhoi). They don’t report what fraction of the cost the tolls will cover. Greater Auckland looked at the NZTA consultation papers about the tolling and say 35 years of tolling will raise $391m. That would be nearly 10% of the (phase 1) cost if you didn’t include interest; it’s a much smaller fraction when you do. And this is phase 1 — there are two more phases in the planned road to Whangārei.  Whether the road is worth the cost isn’t my specialty, but it’s a lot of cost.
• Len Cook (former Government Statistician) is in the Otago Daily Times disapproving of the planned removal of the census enumerations. We’ve covered this topic before.  The changes to the Data and Statistics Act are up for public comment, as are the necessary changes to the Electoral Act.   The electoral changes are not intrinsically controversial but are needed because electoral redistricting is currently triggered by the census. The electoral changes are important because they need a 75% supermajority in Parliament.
• RNZ reports on an NZTA report on public consultation about road changes in Wellington. First, the usual whinge: please link to this sort of report, so we can read it if your summary gets us interested!  Second, and the StatsChat motivation, the NZTA report displays pretty graphics of the public feedback, which are systematically wrong! For example, on the question “will a second Terrace Tunnel make things worse or better for you?” the lower bar is from the report and the upper bar is correct based on the percentages.  The right end of the bar is “better”, and is exaggerated
  
  Or the next question, about Te Aro improvements (original above, correct version below). Again, the “better” end is exaggerated
  
  I don’t think this is likely to be deliberate, but it’s a bad look

The Ministry of Transport have put up a fuel monitoring dashboard. It shows estimates of demand, supply, and price.

At the moment, the reduction in demand is less than 10%, a level of demand that’s probably not sustainable in the medium when global supply is down at least 25%. On the other hand, we are still at level 1 of the alert system, and even level 2 doesn’t ask for any real reductions in demand.

What this display doesn’t show is any sort of “time to running out”.  That’s probably sensible, because it’s not even well-defined, let alone predictable. If you define “running out” as some petrol stations being out of supplies then it’s already happened. If you define it as “no fuel in the country”, it probably won’t happen. And if you define it as level 3 or level 4 restrictions on supply then it’s a choice by the government based on unknown criteria, and so is hard to forecast statistically.

Team Ratings for Week 15

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 112 matches played, 77 of which were correctly predicted, a success rate of 68.8%.
Here are the predictions for last week’s games.

Predictions for Week 15

Here are the predictions for Week 15. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 21

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 140 matches played, 108 of which were correctly predicted, a success rate of 77.1%.
Here are the predictions for last week’s games.

Predictions for Round 21

Here are the predictions for Round 21. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Week 8

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 35 matches played, 23 of which were correctly predicted, a success rate of 65.7%.
Here are the predictions for last week’s games.

Predictions for Week 8

Here are the predictions for Week 8. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 13

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 60 matches played, 44 of which were correctly predicted, a success rate of 73.3%.
Here are the predictions for last week’s games.

Predictions for Round 13

Here are the predictions for Round 13. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 5

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 32 matches played, 19 of which were correctly predicted, a success rate of 59.4%.
Here are the predictions for last week’s games.

Predictions for Round 5

Here are the predictions for Round 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Week 5

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 28 matches played, 20 of which were correctly predicted, a success rate of 71.4%.
Here are the predictions for last week’s games.

Predictions for Week 5

Here are the predictions for Week 5. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

There have been two stories just recently in the Guardian about the dangers of opt-in surveys.  A survey from the respectable polling organisation YouGov reported a big increase in (Christian) church attendance among young people.  This was a bit of a  surprise, and didn’t seem to match up with other polling data (or with attendance counts by denominations that count attendance), but it was YouGov and it was what some people wanted to hear.

Apparently the problem was opt-in respondents.  This isn’t the completely useless opt-in clicky polls that our news sites put up from time to time; YouGov is a serious polling organisation.  However, I think it’s fair to say YouGov has tried to get accurate poll results by focusing more on statistical modelling of who responds and less on trying to get a good sample.  Again, that’s a perfectly reasonable strategy and has historically been competitive. You can’t get real random samples of people any more — not like in the 1950s — and so you get samples that are representative in some qualitative sense and reweight them to match the groups you’re trying to study.

You might think it’s strange that people would try to get into survey samples. It is strange, and that’s exactly the problem. Only a small fraction of people will try to get into surveys for the money, so those people are very unrepresentative, and while they are only a small fraction of the population that’s still a lot of people.   In the future, there’s the potential for LLM-based fake people to take surveys for the money (or just to be inconvenient), and they will be still worse.

When you start with a reasonably well-controlled sample and some people opt out, you have a subset of a reasonably well-controlled sample. It looks as though allowing too much self-selection can be qualitatively worse (though this is a one-off so far, and only provides limited evidence).

I also want to note that CNN reports a response from the Bible Society to the withdrawal of the survey report

The Bible Society said in a statement it was “deeply disappointed” by what had happened, but insisted the “wider picture” from other surveys pointed to “an increased engagement in faith among young adults compared to older generations.”

This isn’t a good reaction: the reason we found out the report was inaccurate was precisely that other evidence didn’t point the same way.

Via Russell Brown on Bluesky, the Herald has a report on the increases in people being charged with cannabis possession. Charges fell by about 1/3 from 2017 to 2021, in parallel with increasing evidence that arrests for possession didn’t really have social license, but then started rising and now are back at nearly 2017 levels.

So what do the police say? Well, the Herald reports

Director of the National Organised Crime Group Detective Superintendent Greg Williams says wastewater testing in the Auckland and Northland region shows cannabis consumption spiking in July 2024.

“If you look at that charging data, it actually perfectly almost reflects what looks like a significant increase in cannabis consumption.”

We can look at the charging data, and the Herald does. We can’t look at the wastewater cannabis data, though.  On the same day in the Herald there was a story on the newest results from wastewater drug analyses. The story reported estimates of meth, MDMA, and cocaine use. As expected, there’s a lot more meth than anything else, but there’s a potentially worrying increase in cocaine (it’s not so much that cocaine is worse than meth, but it’s a new supply chain).  There was no comment in the story on cannabis use.  There were related stories at One News and RNZ and Newstalk ZB.

If you go to the NZ Police webpage on wastewater drug testing you see

The drugs tested for include methamphetamine, MDMA, cocaine, fentanyl, and heroin. These reports focus on methamphetamine, MDMA and cocaine as these drugs are routinely detected by the programme.

At PHF Science (former ESR) you can find plenty of pages talking about their efforts in testing for meth, MDMA and cocaine, such as this one on the 2024 spike in meth, or this research paper with the mind-numbing details of how they do the testing, or this drug harm page where they say

To date, wastewater testing has been used to measure consumption of illicit drugs including methamphetamine, MDMA, cocaine, heroin and fentanyl. 

Neither the police nor PHF Science publish cannabis-use estimates from wastewater.  The reason they don’t publish the estimates is they aren’t very good.  According to a research report from PHF Science,

However, certain characteristics of cannabis – such as it being lipophilic, not dissolving well in water and its tendency to stick to surfaces such as wastewater pipes – have made analysis in wastewater more difficult. Additionally, due to the considerable chemical differences between cannabis and the other illicit substances being monitored it cannot be added to the same analysis workflow. At this stage there is still too much uncertainty for cannabis measurements to be reliably quantifiable. However, the monitoring data can still be used in trend analyses

They do measure cannabis at five sites around the country, and as the research report says, the data could still be used in trend analyses. But popping up with a claim about two regions from undisclosed data about one time period isn’t a credible trend analysis.

What other data are there?

I don’t find the NZ Drug Trends Survey all that convincing on a detailed level, but its questions asking people who admit to using illegal drugs about which drugs they use should also be ok for trend analyses, and their cocaine reports show a similar trend to the wastewater data. They see a decrease and then increase in daily or weekly cannabis use over the time period we’re talking about, but to a much smaller extent: 68% of respondents at the peak, then down to 57%, then up to 70% for the most recent data. That’s about a 15% decrease and corresponding in regular cannabis use among regular drug users.  Also, a big spike in population cannabis use would increase the number of regular drug users, and show up as a decrease in the proportion regularly using other drugs, which we don’t see.

The NZ Health Survey asks about drug use. The Drug Foundation has collected their data (along with other data sources) and it doesn’t show a pattern anything like the police charging data (click to embiggen, as always)

So, I’m not convinced by the bare assertion that wastewater data show the police are just picking up the same fraction of a varying drug-user population. If the police want to use trends in the cannabis wastewater data to influence public policy they should publish the complete data series, with all the attached caveats from the scientists behind the testing (who I do trust).