Posts filed under General (3148)

I wrote about ordinary sampling variation and ‘rogue polls’ for The Spinoff

In clinical trials we make two important distinctions between measurements. There are ‘hard’ and ‘soft’ outcomes — ‘hard’ ones are objectively and reproducibly measurable, ‘soft’ ones have some subjectivity and observer bias.  There are also ‘surrogate’ and ‘real’ or ‘patient-centered’ outcomes. ‘Real’ outcomes are what we care about; ‘surrogate’ outcomes are things we measure because we can measure them well and we expect them to correlate with real outcomes. Hard and soft outcomes are valuable; real and surrogate outcomes are valuable; you don’t want to confuse them.

There’s a story on NewsHub headlined Bisexual men are real, study finds. One of the researchers had previously doubted this, but has now been convinced, and has paper in PNAS about an analysis combining data from many previous studies.  These studies involve wiring someone’s penis up to detect arousal and then showing him erotic images.  The claim is that these measurements are objective

If men who self-report Kinsey scores in the bisexual range indeed have relatively bisexual arousal patterns, then both Minimum Arousal and the Bisexual Arousal Composite should show an inverted U-shaped distribution across the Kinsey range (i.e., men who self-identify as 0 [exclusively heterosexual] and 6 [exclusively homosexual] should have the lowest scores for these variables; men in intermediate groups should have greater values, with the peak resting at a Kinsey score of 3); the Absolute Arousal Difference should show a U-shaped distribution (i.e., exclusively heterosexual and exclusively homosexual men should have lower values than bisexual-identified men).

The reason for emphasising these measurements is that they doesn’t completely trust self-report (while agreeing it is valuable)

However, because the scale relied on self-reports, results could not provide definitive evidence for bisexual orientation. For example, surveys have shown that a large proportion of men who identify as gay or homosexual had gone through a previous and transient phase of bisexual identification 

I don’t think anyone (whatever their opinion on bisexuality) would deny that men who lie about sex are real. The problem is treating the physical arousal measurements as basically definitive of bisexuality.  In the clinical trials terminology, the arousal measurement is a relatively hard outcome, but it is a surrogate outcome.

With modern data science (and sufficiently dodgy ethics) there would be other surrogate outcomes that someone has probably explored.  Are there a significant number of men on Tinder who swipe right for both male and female profiles?  Are there many PornHub accounts of men who watch both straight and gay porn? Are there men who have shared a one-bedroom home with both men and women over time?  All of these are clearly reductive: they would give you one-dimensional information about bisexuality, but they are measuring different things and there’s no reason to expect they would agree on how common it is.  The same is true for physiological arousal.  Measuring it can be valuable; the demographics of physiological arousal can be a valid area of study; but it can’t answer the yes/no question.

Some men claim to be attracted to both men and women, and behave as if their claims are true. It turns out, according to this paper, that for some of these men the physiological measurements of arousal show the relationships that you’d expect.  If there weren’t any men whose physiological measurements of arousal show those relationships, that would be an interesting fact, but the real question would be why the measurements don’t fit with the phenomenon of bisexuality.  If you think of this paper as just trying to answer a question about physiological arousal then, ok, that’s the question it tries to answer. And in fact one of the researchers is quoted further down in the NewsHub story saying

“It has always been clear that bisexual men exist in terms of self-identity and behaviour, but many, including myself, were sceptical about their ability to be sexually aroused to both men and women.” 

Contrast that, though, with the paper’s “Significance” section, which starts out

“There has long been skepticism among both scientists and laypersons that male bisexual orientation exists.”

Or with the second sentence of the press release:

“The existence of male bisexuality is contested, with skeptics claiming that men who self-identify as bisexual are actually either homosexual or heterosexual.”.

Or with the title of the research paper itself

Robust evidence for bisexual orientation among men

The stretching of the study findings to the headline “Bisexual men are real, study finds” can’t just be blamed on the media.

When we talk about whether Alexander the Great or Shakespeare was bisexual, there are difficulties in even agreeing on the concept over centuries or millennia of social distance.  But I think most people would agree there’s more to the question than what would have happened if you wired them up to a machine and showed them porn.

• Interpreting coronavirus numbers, from Pro Publica
• Pretty good Herald story with terrible headline: Covid-19 coronavirus: Superspreader – woman infects 71 people in 60 seconds in elevator: CDC study. The woman in the elevator wasn’t by any definition a ‘superspreader’. She infected one person.  They don’t link to the CDC study.
• Nice explanation of PCR testing for the coronavirus, from U Otago
• Stats NZ are asking for input on their proposed standards for asking about gender and sex.
• Wikipedia is a good source of information on COViD-19. Mike Dickison talks to The Spinoff about why this is.

Non-representative sampling!

The results are in! #MostLoved #RollTide pic.twitter.com/IacOVUhFHL

— Alabama Athletics (@UA_Athletics) July 14, 2020

Team Ratings for Round 4

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 6 matches played, 5 of which were correctly predicted, a success rate of 83.3%.
Here are the predictions for last week’s games.

Predictions for Round 4

Here are the predictions for Round 4. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 7

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 12 matches played, 8 of which were correctly predicted, a success rate of 66.7%.
Here are the predictions for last week’s games.

Predictions for Round 7

Here are the predictions for Round 7. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 11

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 80 matches played, 50 of which were correctly predicted, a success rate of 62.5%.
Here are the predictions for last week’s games.

Predictions for Round 11

Here are the predictions for Round 11. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 3

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 4 matches played, 3 of which were correctly predicted, a success rate of 75%.
Here are the predictions for last week’s games.

Predictions for Round 3

Here are the predictions for Round 3. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 6

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 10 matches played, 7 of which were correctly predicted, a success rate of 70%.
Here are the predictions for last week’s games.

Predictions for Round 6

Here are the predictions for Round 6. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 10

The basic method is described on my Department home page.
Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 72 matches played, 45 of which were correctly predicted, a success rate of 62.5%.
Here are the predictions for last week’s games.

Predictions for Round 10

Here are the predictions for Round 10. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Time magazine has a new story Popular Heartburn Drugs Linked to Heightened COVID-19 Risk, and it’s on Reuters, so it’s going to spread.  Here’s the preprint, and here’s the press release, which comes from the American College of Gastroenterology, the major professional association in that field and the journal publisher.

The drugs in question are proton-pump inhibitors (PPIs). These inhibit the cellular pumps that push hydrogen ions into the stomach fluid, making it acid. If your stomach acid isn’t acid enough, it’s plausible that the coronavirus could survive and get into your gut, where it will find cells with the receptors it needs to attack.  So the theory is not at all unreasonable.  Kiwis will remember that Michelle Dickinson and Siouxsie Wiles both emphasised the potential risk of coronavirus getting in through your mouth.

The press release says “We have now tested the hypothesis in a rigorous study of more than 50,000 Americans and found it to bear out, albeit in an observational study.”  As you’ve probably guessed, I’m not buying that.

In the research paper, the study is described as

we used data from a population-based, online, self-administered survey of Americans collected from May 3 to June 24, 2020. We collaborated with an online survey research firm (Cint) that recruited a nationwide, representative sample based on U.S. Census data on age, sex and region

That could work, provided it really was a representative sample, and provided you could get a good handle on the other reasons why taking a medication long-term might be correlated with getting a positive COVID test.

The sample of 54,000 people included 3,386 (6.4%) who reported having had a positive COVID test.  As of June 24, only 2.35 million people in the US had tested positive for COVID. There are about 250 million adults in the US, so even if all the positive tests had been in adults, that’s less than 1%.   The sample has over six times the national average for positive COVID tests. Regional bias wouldn’t be enough to explain this: even in New York City, only about 2.5% of the population has tested positive.

Even with this very high rate of COVID, the sample is missing a lot of cases. 95% of people with positive tests reported symptoms. That’s not surprising, as symptoms are how you get tested, but it does mean the ‘control’ group will contain at least another few thousand cases who didn’t get tested, or got tested at the wrong time.

If you just look at the PPI data, 75% of people with a COVID diagnosis were regularly taking PPIs, as were 30% of the other participants.  In a population-based health survey that makes serious efforts to be representative, NHANES, only 8.7% of participants reported taking PPIs.

So that’s the overall representativeness. The other thing to worry about is that regularly taking a medication and getting access to a COVID test may well be correlated, so you’d worry about whether the COVID cases were different in other ways — remember, we already know sampling was far from representative.

A few differences jump out. The COVID cases were more than 8 times as likely to have a household annual income over $200,000: nearly two-thirds of them did. That’s despite them being less likely to have a college degree.  More than two-thirds of the cases reported Latinx/Hispanic ethnicity, but only 3.5% were non-Hispanic Black. Only 10% of cases were from the Northeast of the US, where the epidemic has been worst until recently;  nearly 70% were from the South.  The cases were much less likely to report a diagnosis of gastroesophageal reflux disease, a primary reason for taking PPIs.

The researchers did make some efforts to adjust for the non-representative sampling.  The relative risk of COVID in people taking PPIs daily or less often went down from nearly 8 before adjustment to 2.15. For people taking PPIs twice a day, the relative risk went down from 5.7 to 3.7.  However, the researchers didn’t use a lot of the variables in this adjustment, and they didn’t try to reweight the data to known national proportions for age, race/ethnicity, gender, and region, a fairly standard technique in national surveys (eg, election polls, market research).

Given the clearly non-representative sample, I don’t think the evidence could be convincing without a lot more exploration of the biases (and quite likely not even then).  As a drug class, PPIs have form for this: there have been other conditions correlated with PPI use in initial reports, where the correlations go away with better-quality data.

I’m not saying the study shouldn’t have been done, though I think it should have been analysed better. But the journals shouldn’t have pushed it out into the media with an urgent pre-publication press release, especially when even the authors won’t publicly claim it’s good enough evidence to change treatment.  The true take-home message of this study, apparently, is that it gives doctors an opportunity to stress the importance of hand-washing.

If I were more cynical than I am, I would have pointed out much earlier in this post that one of the authors of the study  is co-Editor-in-Chief of the journal, and is quoted in the press release with that title.