Posts written by Thomas Lumley (2646)

• Newshub has a story about so-called ‘lucky’ Lotto stores.  I’ll recycle a previous response.
• The Productivity Commission are arguing that the extra week in lockdown was unnecessary and very expensive. Their analysis is wrong; it does not seem to consider whether and how much the extra week reduced the risk of needing a second lockdown, which was part of the reason for doing in.  I’m not saying the extra week was the right decision — you can’t tell, without modelling the extra risk, which they didn’t do.  It’s like saying insurance is not cost-effective because your house didn’t burn done. Insurance may or may not be cost-effective, but that isn’t how you tell.
• Ed Yong at the Atlantic, on why there’s so much we don’t know about COVID immune response: Immunology Is Where Intuition Goes to Die
• The Human Gene Nomenclature Committee has changed the names of a bunch of genes. Not because they’re named after unpleasant historical figures, but because Excel keeps trying to turn them into dates:  SEPT1, OCT4, MARCH1.  Spreadsheets are useful (and Excel is the world’s most popular statistical software), but you do need to keep a sharp eye on them
• Newshub reports on an attempt to get Pharmac to pay for a drug that costs half a million dollars per patient per year.  I’ll outsource the basic statistical comparison to Matt Nippert on Twitter — the total cost would be about a quarter of Pharmac’s budget (and I’ll just note that this is slightly more than it spends on cancer.)
• If you thought our Census had problems, look at the US.  The American Statistical Association and the American Association for Public Opinion Research are among the groups who want the data collection extended rather than shortened.

At the moment, NZ policy is to test everyone with suitable symptoms consistent with COVID-19, and potential contacts of cases, but not to go around randomly bothering healthy people for community surveillance.  Looking at some numbers explains why that’s a good strategy, and also gives a way to think about what it takes for other surveillance strategies to be useful.

The first question is how well you do just by testing symptomatic people, when we know some people never develop symptoms, and other people only develop symptoms after passing on the virus.  Professor Nick Wilson and various co-workers^* studied this problem back in May.  They did a lot of computer simulations of what would happen if you introduced one COVID case to ‘an island nation’ where the coronavirus had been eliminated but there was still widespread testing.  Under the assumption that about 40% of cases ended up getting tested, they found that an outbreak had a fifty-fifty chance of being detected when there were only six active cases, but that a reasonable worst case was 50-100 active cases at the time of detection.

You can disagree with the particular assumptions being made (they did this way back in May, the coronavirus equivalent of the Sony Walkman era) but it’s a reasonable ballpark guide.   The idea is that you maybe don’t routinely test absolutely everyone with a cold, but you do test everyone with some (new or worsening) respiratory symptom plus shortness of breath, or fever, or loss of sense of smell, or various other combinations.  We’ve gotten lucky: flu-like illnesses are much less common so far this year than in a usual year, so right now we don’t need to test as many people as they modelled.

So, taking the reasonable worst case, suppose at some point there are 50-100 people out there in NZ with coronavirus and we’ve been unlucky enough that none of them got tested (or a few got tested and the tests were false negatives).  How many random people would we need to test to pick up this outbreak?

Fifty people in NZ is one person in 100,000, so we’d need to test about 100,000 people to have a chance of finding a case.  A simple statistical rule of thumb says that testing about 300,000 people would make us pretty sure to find a case.  Outbreaks grow fast;  if there’s an outbreak with 50 people this week, it had maybe  15 people last week. To get a worthwhile improvement in detecting even the worst outbreaks we’d need to test 100k-300k healthy people each week. That isn’t happening.  Random community testing could be useful if we knew where to look. If we had one suspected case in a town of 10,000 people it might be worth just testing as many people as we could, to try to  get ahead of the contact-tracing process. But if you don’t know where to look there isn’t much point in looking.

Sewage testing is another promising possibility for picking up outbreaks, but these numbers show that it’s not going to be easy.  The testing has to be reliable enough that we’d be prepared to take some fairly major and expensive actions based on finding the virus, but sensitive enough to pick up just 50 or so cases.  It currently isn’t clear whether or not that’s possible, but ESR have the expertise (and some funding) needed to work on the question.  Reliable wastewater testing would be very helpful in the situation we’re now in, where there’s a  suggestion of transmission in NZ but not good evidence — but unreliable wastewater testing would just make things worse.

The take-home message is that we’re probably going to find the next outbreak by testing someone with symptoms. That person might very well have no known contact with international travellers.  If you might be that person, you should call Healthline to ask about getting tested.

^* Statisticians will recognise Matt Parry; any Kiwi who hasn’t been hiding in a cave on Mars with their fingers in their ears should recognise Ayesha Verrall and Michael Baker.

We aren’t getting international tourism at the moment, which is obviously a problem for those working in the international tourism industry¹, and to some extent a problem for everyone because of the hit to the economy.

I saw some speculation on Twitter today about how big international tourism actually is, and about the extent to which NZ tourism expenditure staying in NZ would offset the losses.   Now, there will obviously be gaps, where foreign and domestic tourists don’t do the same things (eg, domestic tourists don’t buy long-distance plane tickets from Air New Zealand), but what about the totals?

Overall, tourism (as defined in the tourism satellite account) brought in $17 billion in the year ending June 2019.  Nearly $4 billion of it was actually international education lasting less than a year, leaving a bit over $13 billion in ‘real’ tourism.  That’s just behind dairy, but roughly equal to meat and wood products together.  The short-term international-education component of ‘tourism’ was a bit a head of fruit exports.

Domestic expenditures on international tourism aren’t completely captured, but the Household Expenditure Survey estimates that all NZ households together spent $2 billion on “overseas accomodation prepaid in NZ” and $4.5 billion on “international air transport” in the year to June 2019.  That’s going to miss food bought overseas and admission tickets to cultural experiences, and some overseas accomodation, but it still looks as though redirecting NZ tourism locally would leave a big hole.  The Household Expenditure Survey does miss out on business travel that isn’t a household expenditure, but it seems more of a stretch that business travel spending will just be redirected to NZ.

¹ I was surprised to find this, in one sense at least, includes me, since international students here for less than 12 months are counted in the ‘tourism satellite account’.

The recent trends in opinion-poll support for the National Party got a lot of attention. That’s because real opinion polls, like those done by Colmar Brunton and Reid Research (and the internal party polling that they tell us about when they think it will help them) are genuine attempts to estimate popular opinion.  You can argue about how good they are — but you can argue about how good they are, there are factual grounds for discussion.

NewsHub ran a bogus online clicky poll with the question Who would you prefer as Prime Minister – Judith Collins or Jacinda Ardern?  Of the people who clicked on the poll, 53% preferred Ms Collins, and 47% preferred Ms Ardern.  Let’s compare that to the two real polls. The 1News/Colmar Brunton poll had

• Jacinda Ardern: 54% 
• Judith Collins: 20%

The 3/Reid poll had

• Jacinda Ardern: 62% 
• Judith Collins: 15% 

Why are these so different from the Newshub clicky poll? The first point is that there’s no reason for them to be similar. Two of them are estimates of popular opinion; the other one is a video game.

On top of that, the question is different.  The real polls are asking who (out of basically anyone) is your preferred PM.  The bogus poll forces the choice down to Ardern vs Collins.  If you supported Simon Bridges or Todd Muller — or Metiria Turei  or Winston Peters — the real polls let you say so, and the bogus poll doesn’t.

Research Association NZ, who are the professional association for opinion researchers in NZ, have a code of practice for political polling (PDF). It’s only binding on their members, but it does have best practice advice for the media, such as using the term “poll” only for serious attempts to estimate public opinion, not for bogus clicky website things.

• The Algorithm Charter has been released. Stories from NewsHub, newsroom, The Guardian, The Register, ZDnet
• Covid-19 in Victoria: bad, and according to modelling by Peter Ellis, still getting worse.  If you’re in NZ, make sure you have a mask and hand sanitiser available and at least have the contact apps on your phone, in case we get another outbreak.
• RadioNZ’s podcast “The Detail” has an episode on polling.  I’m reliably informed that I’m on it.
• In 2020, we’re amid that critical juncture for ’90s music—we can finally start asking today’s teens, “What music do you recognize from the ’90s?”. From pudding.cool

A company called Genderify has what they say is “an AI-powered tool for identifying the gender of your customers”. This is an example of something that is not worth doing (asking is easy and reliable; people will be upset when you get it wrong), but also very difficult.

After seeing some examples on Twitter, I decided to try it on some senior members of the Stats department (whose gender identity I’m reasonably confident of)

“Thomas Lumley” is 63.90% likely to be male and 36.10% likely to be female, and you have to like the four digit precision. But “Dr Thomas Lumley” is 89.40% likely to be male, and “prof thomas lumley” gets up to 94.60%!

“Ilze Ziedins” is 85.20% likely to be male, which will surprise her. “Dr Ilze Ziedins” gets to 96.00%

“James Curran” is 99.60% likely to be male, adding “Dr” or “Prof” gets him up to 99.90%

“Rachel Fewster” is at 72.00% likely to be male, adding her professorial title puts that up to 95.40%

“Renate Meyer” is at 62.30%, her doctorate moves that up to 88.20%, and her promotion to professor makes it 94.00%

Note that none of these are classically gender-neutral or gender-ambiguous names: no Hadley or Hilary or Cameron.  The overall level of accuracy is pretty terrible to start with — but the response to adding qualifications is bizarre.  If that wasn’t in the basic pre-release testing, then what was?

Even better (worse):  it’s not just that adding “Dr” or “Prof” make it think you’re more likely mean a man, adding “Dame” also does.

Update: on Twitter, (Dr, Prof) Casey Fiesler raised the possibility that Genderify are just trolling, which I must say is looking quite plausible.

It’s getting to be that time of the triennium again, so some highlights from past StatsChat posts on electoral polling

• Overview of polling and how it works
• What is the ‘margin of error’ for minor parties: one, two
• Why you can’t predict Epsom from polls
• Why polling is important: your friends aren’t representative
• Survey weighting: why polls don’t massively underestimate the Greens vote.
• The difference between 1 in 20 exceedance of the margin of error and a ‘rogue poll‘

I wrote about ordinary sampling variation and ‘rogue polls’ for The Spinoff

In clinical trials we make two important distinctions between measurements. There are ‘hard’ and ‘soft’ outcomes — ‘hard’ ones are objectively and reproducibly measurable, ‘soft’ ones have some subjectivity and observer bias.  There are also ‘surrogate’ and ‘real’ or ‘patient-centered’ outcomes. ‘Real’ outcomes are what we care about; ‘surrogate’ outcomes are things we measure because we can measure them well and we expect them to correlate with real outcomes. Hard and soft outcomes are valuable; real and surrogate outcomes are valuable; you don’t want to confuse them.

There’s a story on NewsHub headlined Bisexual men are real, study finds. One of the researchers had previously doubted this, but has now been convinced, and has paper in PNAS about an analysis combining data from many previous studies.  These studies involve wiring someone’s penis up to detect arousal and then showing him erotic images.  The claim is that these measurements are objective

If men who self-report Kinsey scores in the bisexual range indeed have relatively bisexual arousal patterns, then both Minimum Arousal and the Bisexual Arousal Composite should show an inverted U-shaped distribution across the Kinsey range (i.e., men who self-identify as 0 [exclusively heterosexual] and 6 [exclusively homosexual] should have the lowest scores for these variables; men in intermediate groups should have greater values, with the peak resting at a Kinsey score of 3); the Absolute Arousal Difference should show a U-shaped distribution (i.e., exclusively heterosexual and exclusively homosexual men should have lower values than bisexual-identified men).

The reason for emphasising these measurements is that they doesn’t completely trust self-report (while agreeing it is valuable)

However, because the scale relied on self-reports, results could not provide definitive evidence for bisexual orientation. For example, surveys have shown that a large proportion of men who identify as gay or homosexual had gone through a previous and transient phase of bisexual identification 

I don’t think anyone (whatever their opinion on bisexuality) would deny that men who lie about sex are real. The problem is treating the physical arousal measurements as basically definitive of bisexuality.  In the clinical trials terminology, the arousal measurement is a relatively hard outcome, but it is a surrogate outcome.

With modern data science (and sufficiently dodgy ethics) there would be other surrogate outcomes that someone has probably explored.  Are there a significant number of men on Tinder who swipe right for both male and female profiles?  Are there many PornHub accounts of men who watch both straight and gay porn? Are there men who have shared a one-bedroom home with both men and women over time?  All of these are clearly reductive: they would give you one-dimensional information about bisexuality, but they are measuring different things and there’s no reason to expect they would agree on how common it is.  The same is true for physiological arousal.  Measuring it can be valuable; the demographics of physiological arousal can be a valid area of study; but it can’t answer the yes/no question.

Some men claim to be attracted to both men and women, and behave as if their claims are true. It turns out, according to this paper, that for some of these men the physiological measurements of arousal show the relationships that you’d expect.  If there weren’t any men whose physiological measurements of arousal show those relationships, that would be an interesting fact, but the real question would be why the measurements don’t fit with the phenomenon of bisexuality.  If you think of this paper as just trying to answer a question about physiological arousal then, ok, that’s the question it tries to answer. And in fact one of the researchers is quoted further down in the NewsHub story saying

“It has always been clear that bisexual men exist in terms of self-identity and behaviour, but many, including myself, were sceptical about their ability to be sexually aroused to both men and women.” 

Contrast that, though, with the paper’s “Significance” section, which starts out

“There has long been skepticism among both scientists and laypersons that male bisexual orientation exists.”

Or with the second sentence of the press release:

“The existence of male bisexuality is contested, with skeptics claiming that men who self-identify as bisexual are actually either homosexual or heterosexual.”.

Or with the title of the research paper itself

Robust evidence for bisexual orientation among men

The stretching of the study findings to the headline “Bisexual men are real, study finds” can’t just be blamed on the media.

When we talk about whether Alexander the Great or Shakespeare was bisexual, there are difficulties in even agreeing on the concept over centuries or millennia of social distance.  But I think most people would agree there’s more to the question than what would have happened if you wired them up to a machine and showed them porn.

• Interpreting coronavirus numbers, from Pro Publica
• Pretty good Herald story with terrible headline: Covid-19 coronavirus: Superspreader – woman infects 71 people in 60 seconds in elevator: CDC study. The woman in the elevator wasn’t by any definition a ‘superspreader’. She infected one person.  They don’t link to the CDC study.
• Nice explanation of PCR testing for the coronavirus, from U Otago
• Stats NZ are asking for input on their proposed standards for asking about gender and sex.
• Wikipedia is a good source of information on COViD-19. Mike Dickison talks to The Spinoff about why this is.

Non-representative sampling!

The results are in! #MostLoved #RollTide pic.twitter.com/IacOVUhFHL

— Alabama Athletics (@UA_Athletics) July 14, 2020