Posts written by Thomas Lumley (2645)

From the editing blog “Heads Up”

… you don’t need a course in statistics to ask what a writer means by “incident count,” “city” and “occurrence percentage,” not to mention why and how the means are weighted, or even why users of an insurance comparison website would be a good representation of a city where a huge proportion of drivers are uninsured. 

And

This isn’t “fake news” in the 2016 sense; it’s the old-school kind that has always gotten past enough gatekeepers to do its work. The traditional response is “tell them to buy an ad.”

• “Algorithms can dictate whether you get a mortgage or how much you pay for insurance. But sometimes they’re wrong – and sometimes they are designed to deceive” Cathy O’Neil, for Observer.

• What’s so hard about histograms? A beautiful, detailed exploration by Aran Lunzer and Amelia McNamara

• A talk about human factors research and what it says about data visualisation

• “Point your phone at any mushroom and take a pic, our tech will instantly identify any mushrooms while giving you an article you can read or listen to.” This app seems to be intended as educational ‘augmented reality’, but one reason people want to identify mushrooms is to decide whether it’s safe to eat them. That’s not possible from just a photo, and the costs of some of the possible classification errors are very, very high.

• A new trend in graphics: ‘joyplots’, named for the famous cover art of a Joy Division album. Here’s a history of the album cover, from Jen Christiansen. And now some examples:

Over the years, movies have converged to a length of ~100 min.

4 lines of code with ggjoy. #rstats pic.twitter.com/2xRoRPqVvg

— Claus Wilke (@ClausWilke) July 12, 2017

 

Peak time for sports and leisure #dataviz. About time for a joyplot; might do a write-up on them. #rstats code at https://t.co/Q2AgW068Wa pic.twitter.com/SVT6pkB2hB

— Henrik Lindberg (@hnrklndbrg) July 8, 2017

So, Radio NZ has a story based on a commentary at YaleGlobal on homelessness.

The point of the YaleGlobal piece is that homelessness is increasing as the world gets more urbanised, and that it’s really hard to measure because people define it differently and because some countries don’t want it measured accurately. Overall

Based on national reports, it’s estimated that no less than 150 million people, or about 2 percent of the world’s population, are homeless. However, about 1.6 billion, more than 20 percent of the world’s population, may lack adequate housing.

There’s obviously a lot of room for variation in definitions.

This report isn’t Yale research, really. It’s based on OECD figures, which are reported by governments: the OECD HC3-1 indicator (PDF).  The number for New Zealand is 41705, which we’ve seen last year in the NZ media. It comes from the 2013 census, and was estimated by researchers at Otago.  The NZ homelessness number is high for at least three reasons.  First, NZ uses a very broad definition of homelessness. Second, we’re pretty good at honest data collection. And, third, we’ve got a serious homelessness problem (and have had for a while).

The Government is right to say that the international figures aren’t all comparable. Some countries only count people who are sleeping rough. Others include people in shelters or emergency accomodation. We include a lot more. The Herald story from last year quotes an Otago researcher, Kate Amore

“If the homeless population were a hundred people, 70 are staying with extended family or friends in severely crowded houses, 20 are in a motel, boarding house or camping ground, and 10 are living on the street, in cars, or in other improvised dwellings.”

From that tally, a few countries don’t even count all of the 10; some don’t count all of the 20; many don’t count the 70 —  and some aren’t very good at counting.

Here’s a set of charts I made based on a crude classification of definitions from the OECD HC3-1 report. The numbers on the axis are in % of the population.

Even within the top panel, NZ, the Czech Republic, and Australia have the broadest definitions. The HC3-1 report says

Australia, the Czech Republic and New Zealand report a relatively large incidence of homelessness, and this is partly explained by the fact that these countries adopt a broad definition of homelessness. In Australia people are considered as homeless if they have “no other options to acquire safe and secure housing are without shelter, in temporary accommodation, sharing accommodation with a household or living in uninhabitable housing”. In the Czech Republic the term homeless covers “persons sleeping rough (roofless), people who are not able to procure any dwelling and hence live in accommodation for the homeless, and people living in insecure accommodation and people staying in conditions which do not fulfil the minimum standards of living […]”. In New Zealand homelessness is defined as “living situations where people with no other options to acquire safe and secure housing: are without shelter, in temporary accommodation, sharing accommodation with a household or living in uninhabitable housing.”

I think the New Zealand definition is a good one for measuring the housing deprivation problem, but it’s not good for international comparisons.

On the other hand, the comparison to Australia is pretty fair, and there’s at least no evidence that anywhere else has higher rates.  To some extent we have an apples vs oranges comparison, but that doesn’t stop us concluding it’s a bad apple.

Time magazine’s website has had at least seven stories about avocado on toast since May. The most recent one says

Square, a tech company that helps businesses process credit card payments, crunched data from sellers around the U.S. and found that Americans are spending nearly $900,000 per month on crusty bread topped with mashed green fruit.

What’s impressive about that number is how unimpressive it is.  The US is a big place. Even if we only count millennials, there are 80 million of them, so we’re talking about an average of 1 cent each per month.

But it gets worse from there. The story talks about the 50-fold increase since 2014. That’s an increase in sales handled by Square, an expanding business. Neither Time nor Square seems to have made any attempt to look at sales from comparable businesses over time, which Square could have done easily.

The whole avocado-toast thing only makes sense as a synecdoche for an eating-out lifestyle, so accurate data about avocado on toast isn’t really going to be very helpful for anything important. Even so, it’s possible for innumerate presentations of inaccurate data to be less helpful.

From the Chronicle of Higher Education

Mr. Vijg said repeatedly that his Nature paper made no “definitive statement” about a maximum human age and that he felt “amazement” that anyone might think otherwise. But he acknowledged approving a news release about his study issued by Albert Einstein College with the headline: “Maximum human lifespan has already been reached, Einstein researchers conclude.”

It’s more often the press release than the research paper, but with the consent of the researchers. If press releases were signed by a researcher and made available along with the research paper, there’d be more incentive not to do this sort of thing.

• The Australian has a story about religion in Australia, with the figure of “a touch under 70 per cent nominating a religious affiliation.” That’s only true if you include the 10.6% “Religious affiliation not stated” as nominating a religious affiliation.  The true figure is 56.8%.  You could stretch that to 63.5% of those who answered, but that’s assuming (at least implicitly) that the no-answers are just as likely to be religious as those who stated an affiliation. (via John Quiggin)

• This is why 10 stunning photos will make you cry tears of joy.

• From the Dallas Morning News:Steve Doud, a subscriber from Plano, emailed me to say he’d read something in the June 21 Dallas Morning News that couldn’t possibly be true.An eight-paragraph Washington Post article on page 10A reported on a national study about kids and guns. The last sentence said 4.2 percent of American kids have witnessed a shooting in the past year.

• There’s news in the UK claiming that Ryanair’s “random” seat allocation for people who don’t pay extra to pick a set isn’t random and favours middle seats, including something from Oxford University statisticians. It’s clear that the seats are not a simple random sample of all seats on the plane, but that isn’t what Ryanair is claiming. The airline’s defence, that the people who pay to pick seats usually pick the better ones, seems pretty plausible to me.

• Buzzfeed says the US is going to spend $7.1 billion on food for the Fourth of July weekend. If you think about this as 321 million people for a weekend plus a holiday, it’s not as impressive.  Maybe they’re only counting some types of food?  This violates two of the three basic rules for big dollar numbers: they need to be per-capita, inflation-adjusted, and compared to something. (via Fred Clark)

• “A sloppy attitude towards statistics has led to exaggerated and unjustified claims becoming commonplace in science, according to one of Britain’s most eminent statisticians.” David Spiegelhalter, reported in the Guardian

And finally, a picture of one of the causes of overdiagnosis in cancer — and one of the causes of illusory improvements in survival from earlier screening.

What is overdiagnosis? This refers to the detection of cancers that would never cause harm: https://t.co/tEJ7I54a1z pic.twitter.com/VY8okHOL9q

— National Cancer Inst (@theNCI) June 30, 2017

StatsChat has been going long enough that I can look back at stories I disparaged in the past and see how well they held up.

To begin with, an April 2012 story about a vaccine for heart disease that might be available within five years. It wasn’t.

There doesn’t seem to have been any real development of this vaccine. But there is a 2017  research paper that has a new promising approach to vaccinating against heart disease.

You might have seen a story in the Guardian (don’t click while you’re eating), headlined “Popularity of sushi has brought rise in parasitic infections, warn doctors“.

There’s a link (also not safe for dinner) to a single case report of anisakiasis in Portugal attributed to sushi. The case report says

Most of the cases were described in Japan due to food habits; however, it has been increasingly recognised in Western countries.2 ,3

If you follow those references, you find a report of 2 cases in Italy, due to marinated raw anchovies, and 25 cases in Spain, due to marinated raw anchovies.

Anisakiasis is now being diagnosed in western countries, but if we can believe the evidence behind this story the cases are mostly from traditional European food practices, not because of  sushi.

Q: Did you see cannabis is actually better for pain relief than opioids?

A: Not convinced.

Q: But 92%97% of patients said they preferred it! Even if they’d tried opioids! You’re not saying you know better than them, are you?

A: Imagine you had a survey of people who cycle to work, including lots of people who own cars.

Q: Ok.

A: What proportion of those people would say they’d choose to cycle to work instead of driving?

Q: I don’t know, I don’t drive.

A:  Imagine you had a survey of people who cycle to work, including lots of people who have perfectly good bus routes as alternatives.

Q: Ok.

A: What proportion of those people would say they’d choose to cycle to work instead of driving or taking the bus?

Q: A lot.

A: Maybe even 92%97%?

Q: Maybe.

A: The survey was of about 2000 current medical cannabis users. It’s not surprising they say it works.

Q: So it’s not true?

A: No, I think it quite like is true, or at least partly true — not for everyone, but for a lot of people.

Q: Then what’s the problem?

A: The survey provides almost no additional evidence that it’s true. I already thought it was plausible, but anyone who didn’t think that shouldn’t have had their mind changed by the survey.

Q: Ok, what would convince you? Do we need randomised trials?

A: That would be nice.

Q: And a pony?

A: Ok, ok.  More realistically, a cohort study that follows up a group of people starting to use cannabis for pain relief to find out what they end up preferring, rather than starting with people who prefer cannabis.  You could set that up somewhere were medical use was just being legalised, and it wouldn’t cost that much.

• A map of 1.3 billion taxi trips in New York, taking advantage of the underappreciated principle that there’s no point having more detail than the screen can display.  Also, GPS error naturally gives an attractive glowing effect that you’d usually have to add in afterwards

• “In the summer of 2015, Alexandra Franco got a letter in the mail from a company she had never heard of called AcurianHealth. The letter, addressed to Franco personally, invited her to participate in a study of people with psoriasis, a condition that causes dry, itchy patches on the skin.”  A story about creepy data-mining, from Gizmodo.

• From Scientific American, graphics showing daily, weekly, yearly patterns in number of births.

• From the New York Times: a new drug for muscular dystrophy. It costs about US$1 million per year, and the FDA is not really convinced it has an effect

• It’s time for the NZ Garden Bird Survey, which means it’s time for me to recommend their questions and answers page for its attention to principles of experimental design.

• “Death when it comes will have no sheep”. Last week it was hamster names; this week it’s proverbs. Look, save yourself some effort and just go directly to Janelle Shane’s blog rather than waiting for each post to go viral.

• “In Science, probability is more certain than you think.” Chad Orzel