Stats Chat

If you don’t like StatsChat you will probably not enjoy “More or Less”,  a BBC radio show/podcast/blog on statistics in the British media, presented by Tim Harford.  Their new season starts this week.

According to the Herald

The latest Roy Morgan Poll shows support for the National Party has fallen two per cent since early June.

 The poll is based on 948 people, so the maximum margin of error (which is a good approximation for numbers near 50%) is about 3.2%, and the margin of error for a change between two polls is about 1.4 times larger: 4.6%.

The Herald reports on a study of weight gain after quitting smoking, which is based on analyzing the results of 62 randomized trials of treatments to help quitting.  Ordinarily, data from randomized trials is what we want, so why is Professor Simon Chapman quoted as complaining the results are unreliable?

Well, it’s partly because he doesn’t like anything that might be construed as favorable about smoking, but he has a good point in this case.  Randomized trials give us fair and trustworthy comparisons between two treatments: in this case the 62 trials tell us something about which ways to quit actually lead to the most quitting.   The information on weight gain, on the other hand, isn’t a comparison of two randomized treatments, it’s a before-after comparison on the people who managed to quit.  The fact that the treatments were randomized is of no help at all, since the analysis lumps all quitters together.

In fact, it’s worse than that. A lot of smokers manage to quit with only moderate difficulty.  They tend not to end up in randomized trials.  Some smokers find quitting much harder, and they are much more likely to end up in randomized trials.  So the research actually has found that a group of people who probably found quitting hard have gained 4-5kg after quitting.   It’s quite likely that people who find quitting hard are also going to gain more weight than those who quit without major difficulties, so we may well be overestimating the impact of quitting on weight.

Since I regularly complain about the lack of randomised trials in education, I really have to mention a recent US study.  At six public universities in the US, introductory statistics students who consented were randomised between the usual sort of teaching by real live instructors or a format with one hour per week of face-to-face instruction augmented by independent computer-guided instruction.  Within each campus, the students were assessed in the same way regardless of their instruction method, and across all campuses they also took a standardised test of statistics competence.   Statistics is a good target for this sort of experiment, because it is a widely required course, and the median introductory statistics course is not very good.

The results were interesting.  The students using the hybrid computer-guided approach found the course less interesting than those with live instructors, but their performance in the course and in the standardised tests was the same.   If you ignore the cost of developing the software (which in this case already existed), the computer-guided approach would allow more students to be taught by the same number of instructors, saving money in the long run.

This doesn’t mean instructors are obsolete — people like face-to-face classes, and we do actually care if students end up interested in statistics –but it does mean that we need to think about the most efficient ways to use class contact time.  There’s an old joke about lectures as a method of transferring information from the lecturer’s notes into the students’ notebooks without it passing through the brains of either.  We’ve got the internet for that, now.

Team Ratings for Round 19

Here are the team ratings prior to Round 19, along with the ratings at the start of the season. I have created a brief description of the method I use for predicting rugby games. Go to my Department home page to see this.

Performance So Far

So far there have been 128 matches played, 76 of which were correctly predicted, a success rate of 59.38%.

Here are the predictions for last week’s games.

Predictions for Round 19

Here are the predictions for Round 19

Team Ratings for Week 21

Here are the team ratings prior to Week 21, along with the ratings at the start of the season. I have created a brief description of the method I use for predicting rugby games. Go to my Department home page to see this.

Performance So Far

So far there have been 113 matches played, 80 of which were correctly predicted, a success rate of 70.8%.

Here are the predictions for last week’s games.

Predictions for Week 21

Here are the predictions for Week 21. The prediction is my estimated points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Stuff is linking to a map of earthquakes by John Nelson of IDV Solutions.  Long-term readers may recall my earthquake map, which uses just the earthquakes since 1973, where the data is more complete.   John Nelson’s map is certainly prettier, but I think mine is clearer.

Daniel Kahneman and Amos Tversky made huge contributions to our understanding of why we are so bad at prediction.  Kahneman won a Nobel Prize[*] for this in 2002 (Tversky failed to satisfy the secondary requirement of still being alive).  Kahneman has now written a book, Thinking, Fast and Slow about their research.  Unlike some of his previous writing, this book is designed to be shelved in the Business/Management section of bookshops and read by people who might otherwise be  looking for their cheese.

The “Fast” and “Slow” of the title are two systems of thought: the rapid preconscious judgement that we use for most of our decision-making, and the conscious and deliberate evaluation of alternatives and probabilities that we like to believe we use.   The “Fast” system relies very heavily on stereotyping — finding the best match for a situation in a library of stories — and so is subject to predictable and exploitable biases.  The “Slow” system can be trained to do much better, but only if we can force it to be used.

A dramatic example of the sort of mischief the “fast” system can get up to is anchoring bias.  Suppose you ask a bunch of people how many UN-member countries are in Africa.  You will get a range of guesses, probably not very accurate, and perhaps a few people who actually know the answer.  Suppose you had first asked people to write down the last two digits of their telephone number, or to spin a roulette wheel and write down the number that is chosen, and then to guess how many countries there are in Africa.  Empirically, across a range of situations like this, there is a strong correlation between the obviously irrelevant first number and the guess.   This is an outrageous finding, but it is very well confirmed.   It’s one of the reasons that bogus polls are harmful even if you know they are bogus.

Kahneman gives many other examples of cognitive illusions generated by the ‘fast’ system of the mind.  As with optical illusions, they don’t lose their intuitive force when you understand them, but you can learn not to trust your intuition in situations where it’s going to be biased.

One minor omission of the book is that there’s not much explanation of why we are so stupid: Kahneman points out, and documents, that thinking uses up blood sugar and is biologically expensive, but that doesn’t explain why the mistakes we make are so simple.  Research in computer science and philosophy, by people actually trying to implement thinking, gives one possibility, under the general name of “the frame problem“.  We know an enormous number of facts and relationships between them, and we cannot afford to investigate the logical consequences of all these facts when trying to make a decision.  The price of tea in China really is irrelevant to most decisions, but not to decisions about tea purchases, or about souvenir purchases when in Beijing, or to living-wage levels in Fujian.  We need some way of ignoring the price of tea in China, and millions of other facts, except very occasionally when they are relevant, without having to deduce their irrelevance each time.  Not surprisingly, it sometimes misfires and treats information as important when it is actually irrelevant.

Read this book.  It might help you think better, and at least will give you better excuses for your mistakes.

* to quote Daniel Davies: “blah blah blah Sveriges Riksbank. Nobody cares, you know.”

Congratulations to Tony Cooper for his nomination of “Men over 50 nation’s biggest drinkers”, which Thomas Lumley picked up on and posted about as a result.

It’s a great example of how the media can get it so wrong – and in doing so has created a myth. Congratulations Tony!

NBR’s editor Nevil Gibson links up to Stats Chat today in his discussion of the minimum price of alcohol:

In StatsChat’s brilliant daily skewing of journalistic bloopers, University of Auckland Biostatistics Professor Thomas Lumley says this is the opposite point of minimum unit pricing, as opposed to increased excise rates.

Read the full editorial »