Posts filed under General (3152)

You’ve probably heard of OkCupid, a dating site. People give sites like that a lot of personal information. And, in a sense, the information is obviously not going to be kept secret — after all, the point of using a dating site is to be found by people you don’t already know.  When someone writes a script to collect the data from large numbers of users, and then publishes it in a convenient and easy to process format, you can just about see how they’d think that was ok. It’s harder to see how they’d be surprised not everyone feels that way.

Aggregation makes a difference because we can search, match, and analyse the data by computer. That’s important for two reasons.

First, it’s quicker and easier — you can get a set of records grouped by sexual preference or other interests almost as quickly as you can think of the question, and you can link usernames or other information to other datasets. The database includes potential matching variables like income, education level, age, job, country, city, which you could still use just taking down data one person at a time by hand, but it would be slow and boring.

Second, the database is impersonal. If you stood outside a gay bar watching who went in and out, you couldn’t really pretend you were innocently using publicly visible information.  If you signed up and went through dating profiles one at a time, it would be easier to pretend, but you’d still tend to see the people behind the data. When it’s a big spreadsheet, it’s easier to ignore how the people would feel about it.

Sometimes people aggregate and publish data knowing it may do harm, because they think there’s a higher interest involved in getting the data out — even if the data release is obviously illegal. This release isn’t obviously illegal (though there are possibilities), but the higher interest is pretty obscure too. The accompanying research paper says

As an example of the analyses one can do with the dataset, a cognitive ability test is constructed from 14 suitable items. To validate the dataset and the test, the relationship of cognitive ability to religious beliefs and political interest/participation is examined.

Those variables are so not what’s going to attract people to these data. But even if you think it’s important for anyone on the internet to be able to do that sort of correlation for variables such as sexual orientation and drug use, it’s hard to think of a reason to include the OkCupid username.

Q; Did you see yoghurt prevents cancer?

A: Where?

Q: The Herald (from the Daily Telegraph): “8 ways to lower your cancer risk.” Number one is “Eat yoghurt”. And they even have a link to research. How’s that for impressive?

A: Not exactly a link. They mention the name of a journal, but don’t even give the researchers’ names.

Q: Can’t you find them?

A: Of course. It’s even open-access.

Q: So, how much yoghurt did the people have to eat?

A: No yoghurt was harmed in this experiment. Also no people.

Q: Mice?

A: Mice.

Q: But yoghurt?

A: No. Some of the mice were set up with a restricted set of gut bacteria (missing known nasty ones) by being raised in a mouse colony who all had the restricted set.

Q: But the story says “gave one group of mice beneficial bacteria through probiotic supplements and the other non-beneficial bacteria. ”

A: Yes, it does. The research paper, not so much. Nor even the press release.

Q: So why yoghurt?

A: One of the bacteria that was more common in the mice with the restricted set is a Lactobacillus strain. Other Lactobacillus strains, even sometimes from the same species, are involved in making yoghurt, sourdough, sauerkraut, kimchi, etc.

Q: And you could use the mouse bacteria to make these foods?

A: In principle, probably, though you might not want to advertise it that way.

Q: So, the mice with more Lactobacillus were less likely to get cancer?

A: These were mutant mice who all get cancer, so that’s not really the question. They took longer to get cancer.

Q: So we can’t really be confident yoghurt would prevent normal mice from getting cancer?

A: No, it’s too soon to tell.

Q: Good thing normal mice don’t read the newspapers, then.

Team Ratings for Round 12

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 85 matches played, 58 of which were correctly predicted, a success rate of 68.2%.
Here are the predictions for last week’s games.

Predictions for Round 12

Here are the predictions for Round 12. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

The first data under the new real-estate ownership reporting scheme is out. The Herald has a story and also includes a full copy of the report.

So, what proportion of Auckland property sales were reported as being to China?

In Auckland, the level of foreign investment was slightly higher than the national level, at 4 per cent, or 474 properties. Nearly 60 per cent of these properties went to Chinese tax residents.

That’s 60% of 4%, or a bit under 2.5%.  Auckland is different; in the rest of New Zealand the majority of foreign (tax-residency) investors are Australians.

The LINZ report does a good job explaining the real limitations of `tax residence’ as a criterion, but it’s a lot better than any previous data we’ve had.

There were also questions about actual residency and intention to occupy a home, but these were harder to interpret because of property bought by companies or trusts, where the questions didn’t have a good answer.

I’d suggest starting with the report rather than the news coverage.

“Coffee today is like God in the Old Testament”, says John Oliver, reviewing the positive and negative headlines over the past year or so.  It’s excellent, if a little overblown in places.

On a related note, another site has fallen for the ‘cheese addiction/casomorphin’ hoax that we’ve seen before a few times. This time it’s Pharmacy Times.

• Interactive exploration of colours used to label subway lines around the world

• A historical list of data visualisations, starting in 5500BCE Mesopotamia

• An example of rainbow colour scales actually misleading people

• Selection bias: @_OneRandomTweet retweets one random tweet every few hours, “to remind you that people don’t use Twitter like you do“

• New Zealand used to have open data on road crashes. Richard Law writes about why getting rid of it was bad.

• Richard Clark: “The boundaries of New Zealand suburbs and localities is held by the New Zealand Fire Service. For years, the NZFS has refused to provide this data under any terms except a restrictive license, and it has to stop.“

• Outsourcing: Andrew Gelman is disappointed in the NZ Herald, so I don’t have to be.

• Survivor bias:

pic.twitter.com/3LS7yH8adC

— Persian Rose (@PersianRose1) April 24, 2016

Team Ratings for Round 11

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 78 matches played, 53 of which were correctly predicted, a success rate of 67.9%.
Here are the predictions for last week’s games.

Predictions for Round 11

Here are the predictions for Round 11. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Team Ratings for Round 10

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 72 matches played, 40 of which were correctly predicted, a success rate of 55.6%.
Here are the predictions for last week’s games.

Predictions for Round 10

Here are the predictions for Round 10. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

From NewsHub: “NZ’s most violent city spots revealed”

The approximately one square kilometre grid follows part of Queen St and includes the area around the Sky Tower and casino, as well as the eclectic entertainment strip of Karangahape Rd.

Last calendar year 550 people were the victims of assaults, sexual attacks and robberies in this area.

That’s a rate for these violent crimes more than six-and-a-half times the national average.

The other top locations included two more areas in central Auckland, and a chunk of central Wellington including Cuba St and Courtenay Place.  One thing these four (and quite possibly some of the other top locations) have in common is that a lot of people who don’t live there spend time there — and some of these people commit or suffer violent crimes.  Auckland Central West has a very high violent crime rate for its local population, but some of that is because the relevant population isn’t just the local residents, it’s workers by day and revellers by night.   The area is presumably more dangerous than the national average, but it’s not six and a half times more dangerous.

Team Ratings for Round 10

The basic method is described on my Department home page.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Performance So Far

So far there have been 70 matches played, 46 of which were correctly predicted, a success rate of 65.7%.
Here are the predictions for last week’s games.

Predictions for Round 10

Here are the predictions for Round 10. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.