Stats Chat

This summer, we have a number of fantastic students who received a Department of Statistics scholarship to work on fascinating projects with our staff members. We’ll be profiling them here on Stats Chat and we’d love to hear your feedback on their projects!

Joshua Dale is working with David Scott on sports prediction.

Joshua explains:

“David Scott has been predicting the outcome of rugby union and rugby league games using an exponential smoothing method. The predictions have been posted on Stats Chat and his Super 15 predictions have also appeared in the New Zealand Herald. Whilst the predictions have been quite successful and David was equal best amongst the NZ Herald tipping panel in predicting Super 15 games in 2012, it is likely that the method can be improved.

“The project will investigate some possible improvements:

1. The use of more parameters for home-game advantage;
2. The use of a power transform of the prediction errors used for updating team ratings;
3. Adaptive estimation of the smoothing parameter.

“If time permits, the problem of automatic updating of fixture lists from websites will also be considered. The data analysis and optimisation will be primarily carried out using the statistical programming language R. I have taken a couple of computer science papers and I use R a fair bit in statistics courses, which will be a big help with this project.”

More about Joshua:

“I’m just about to finish a Bachelor of Commerce degree where I majored in finance, but also studied several statistics papers as well, enough to gain entry into the Bachelor of Science (Honours) programme in statistics for 2013. During 2012, I had the opportunity to work for the Department of Statistics, both marking assignments and tutoring in the computer labs. This has been an incredibly worthwhile experience, and I plan to do it again next year.

“As a lab demonstrator, students ask you for help with concepts and assignments. I feel because the tutors in the labs are students themselves, they are able to explain concepts in a way that the students can relate to, which makes their learning experience much more enjoyable.

“I like statistics because of its applicability to a huge number of problems. In almost any situation where data is involved, statistics can be used to increase efficiency, improve profitability, make predictions, and help to provide insight into many other areas. It’s also reassuring, in terms of job prospects, that with the heavy use of computers and the internet today, corporations are collecting more data than ever before. Somebody’s got to analyse it!

“In addition to focusing on this project, over summer I will be learning how to program in SAS (a major commercial statistics package), studying for the CFA (Chartered Financial Analyst) exams, and trying to find undervalued stocks to buy on the New Zealand and Australian stock markets. You’ll also find me mountain biking on the weekends, building a petrol-powered bike in the garage, and heading to the Coromandel for New Year.”

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday December 14 2012.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of December 8 – 14 2012 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

This summer, we have a number of fantastic students who received a Department of Statistics scholarship to work on fascinating projects with our staff members. We’ll be profiling them here on Stats Chat and we’d love to hear your feedback on their projects!

Liza Bolton is working with Mark Holmes on an interactive simulation of interacting particle systems.

Of the scholarship, Liza says:

“I am delighted to have this opportunity over the summer, as I’m in love with mathematics and statistics (my two majors in a Bachelor of Science) and was keen to sink my teeth into learning Java. The opportunity to solve problems, sometimes completely abstract, and sometimes with very relevant practical importance, has always drawn me to these subjects. I’ve just completed the second year of my BSc and hope to go on to Honours after my undergrad, before working for a while, and then probably finding my way back to further study.”

Liza explains what the research project is about:

“Imagine you’re on a balcony, looking down on a crowded public square. Below you there’s a sort of multi-party political expo going on. The east wall has a line of Labour Party supporters against it. There are National Party supporters lined up against the west wall, and a line of Green Party supporters on the north wall.

“Do you have that picture in your head? Now, these people lining the walls of the square are never going to change their mind about who they want to vote for. But next, imagine that all the other people in the square (it’s absolutely packed, people are shoulder to shoulder) are far more malleable in their political views. In fact, each person might change their opinion based on the people around them. To stretch the allegory just a little, you can tell that all this is happening from your perch up on the balcony because people are wearing coloured hats to show their party choice, and change to a hat of their new party’s colour when they change their mind.

“Now, political opinion is more complicated than the scenario above, but I hope this might help you picture the actual process of what I’m getting up to this summer. I’m working on creating an interactive simulation of interacting particle systems with Dr Mark Holmes.

“Instead of people, in a square, with peculiar party-appropriate headwear, think instead of a grid with each square (or particle) in the grid a colour. In each time step, one of the squares in the grid is randomly selected to change colour, based on the colours of the squares with an edge adjacent to it. If a blue square is going to change in a particular time step, and has two red and two green squares next to it, it is with probability 0.5 that it will change to green, and likewise to red. And just as with the people you can choose to line the sides of the grid with squares that will not change colour.

The goal for this research is:

“To create a Java applet that people can interact with online that exhibits this idea and also communicates what is going on in a way that will be widely accessible. By the end people will be able to play with this applet, select the colour and position of invariant squares however they like, and watch the system progress through time. Hopefully, it will be a fun little way to create patterns and watch them form, and allow for a basic introduction to the complex and fascinating world of interacting particle systems.”

When Liza’s not going square-eyed staring at grids this summer …

“… I’ll be wearing the hat of Chief Operations Officer for P3 Foundation, a completely youth-lead non-governmental organisation working to empower Kiwi youth to eradicate extreme poverty in the Asia-Pacific. I’ll be doing a bit of work for Teach First NZ, an organisation working to tackle educational inequality in New Zealand, and the family holiday programme at the Voyager Maritime Museum, too. And of course, eating ice cream, reading good books, watching Vlogbrothers and hanging out with my lovely whānau.”

The United States National Oceanic and Atmospheric Administration (NOAA) released a usual monthly State of the Climate Global Analysis in October 2012 which was anything but usual. Buried in the report was this astounding factoid: “This is the 332nd consecutive month with an above-average temperature”. What that means is, if you’re 27 or younger, you’ve never experienced a colder-than-average month. I find that phenomenal, a trend which if it continues might allow many to become ‘old timers’ who can “recollect the ‘good old days’ when a monthly temperature could be below average” (i.e. prior to March 1985).

This statement is certainly headline-grabbing, although there is some devil in the detail. Specifically, what is the ‘average’ which the NOAA benchmark against? A bit of research reveals that the NOAA use a reasonably robust 3-decade (1981-2010) average for their graphics, but the phrasing of the paragraph in question suggests that in this case they are comparing to the 20^th century average. If it was the 3-decade dataset then the months Jan 1981 – Feb 1985 would have had to have been exceptionally cold to skew the average so low that it could be exceeded 332 times consecutively.

The field of time series and calculating moving averages (and variabilities) is fascinating, and no doubt with sufficient data-mining, as in any field (imagine sporting statistics), a number of other shocking statistics could be extracted. Nonetheless a 332 month run is still impressive (or incredibly concerning). Interpretation of climate change will more and more require punters to be comfortable with interpreting both running averages and changing variabilities. We can have extremely cold snaps in a month (variability) while still having an above average month for temperature.

From the Tow Center for Digital Journalism, at Columbia University, a new report: “Post Industrial Journalism: Adapting to the Present”

• Journalism matters.
• Good journalism has always been subsidized.
• The internet wrecks advertising subsidy.
• Restructuring is, therefore, a forced move.
• There are many opportunities for doing good work in new ways.

Stuff tells us

Polls have Labour closing in on Nats

A One News-Colmar Brunton poll released last night and taken a week after Labour’s leadership spat, saw the party’s vote lift 3 percentage points to 35 per cent, with the Greens up one on 13 per cent.

and later on

The One News poll of 1000 had a margin of error of 3.1 per cent.

Did they have another poll? Well, the story goes on to say

 Meanwhile, a 3News-Reid Research poll, also released yesterday, showed Labour on the rise at 34.6 per cent, up 1.6 percentage points. The Greens improved 1.3 to 12.9 per cent, while National was down 1.8 at 47 per cent.

It doesn’t make as good a lead, because Labour was only up 1.6%, not 3% in this poll.  But if there are two polls, perhaps there are even more out there.

The poll of polls at pundit.co.nz shows a steady trend towards higher support for Labour, and a trend towards lower support for the Greens (with a bit more variation around the trend).  Their last update (including these two polls) put Labour up by 1% and Greens up by 0.2%.  But that would be less newsworthy.

In another story bemoaning the affordability of housing outside Auckland, the Herald also tells us

The median sale price of a home in central Auckland is now $690,000 – slightly up on the previous quarter and 12.5 per cent on the 2007 boom.

That’s still down on the 2007 boom in real terms.  The story was followed the very next day by

Auckland real estate asking prices fell 2 per cent last month

The signal to noise ratio in real-estate industry press releases is not all that high.

Congratulations to Eva Laurenson for her excellent nomination of the NZ Herald’s article entitled “Manukau ‘luckiest’ place for Lotto”:

What does ‘luckiest’ in this title mean? Well to the average person ( I asked a few) they interpreted that title as ” I would have a higher chance of winning Lotto if I bought my ticket from a Manukau store compared to another store from a different suburb in Auckland.” Is this really the case? I doubt it. The article ranks Manukau ‘luckiest’ because it is the suburb with the highest total paid out first division amount. However no where did they take into account the total sales of Lotto tickets in each suburb. I think if you took this into account you’d see that Manukau sells alot more tickets than some of these other suburbs in Auckland. So even though Manukau can boast 55 mil in first division prizes we have no idea whether that is 55 mil out of 100 mill worth of ticket sales or 55 mil out of 1 bill worth of ticket sales. Some of the other suburbs may have a lesser amount of first division payouts compared to Manuaku but could have a greater proportion of first division payouts compared to ticket sales. Hence if that was true, your chance of winning first division given that you bought your ticket in that other suburb would be greater than (the same probability measured for) Manukau. Therefore I think there isn’t sufficient information provided to make this claim.

What I think the article could say is ‘given I won first division, the chances that I bought my ticket in Manukau are ____ times the chance that I bought it somewhere else.’ Something to this effect could be derived from the information presented by the herald article and it makes a bit of sense. Is this what the article wrote though? Not at all. They summarised this finding into “Manukau is the luckiest Lotto suburb in Auckland.” Please! This screams misleading. As discussed above, there simply isn’t enough information to justify labelling Manukau the ‘luckiest’ suburb for Lotto. People have a clear idea of what it means to be lucky and that generally is that they have an increased chance of winning. This is not the conclusion you can draw from the information they provided and in this case I believe the herald got it wrong.

I also think, although probably not the authors intentions, labelling Manukau as the ‘luckiest’ suburb has the danger of enticing people to spend more on Lotto. This article published earlier in the year by the NZ herald noted that “Many South Auckland suburbs featured among those which gambled away the most money. Mangere Bridge, Flat Bush, Manukau and Manurewa were in the top dozen suburbs.”
Even though the article was talking about the pokies, Lotto is just another form of gambling. We shouldn’t be condemming one and sending a rosy message about another, especially to communities who are struggling as it is.

Overall I think this should be the Stat of the week because using ‘lucky’ was a nice little pun but in effect mislead people regarding their chances of winning first division depending on where they bought their ticket.

Secondly it seems wrong to label a suburb ‘luckiest’ and potentially encourage a community to spend more on Lotto there when it is known that it is a compartively poorer area than other Auckland suburbs and spends alot of money on gambling as it is.

Thomas expanded on this, saying:

This looks as if it’s claiming that tickets bought in Manukau have been more likely to win. If this was true, it would still be useless, because future lotto draws are independent of past ones.

It’s even more useless because there is no denominator: not tickets sold, not people in the suburb, not even number of Lotto outlets in the suburb.

What the statistic, and the accompanying infographic, really identifies is the suburbs that lose the most money on Lotto. That’s why Manukau and Otara are ‘lucky’ and Mt Eden and Remuera are ‘unlucky’, the sort of willfully perverse misrepresentation of the role of chance that you more usually see in right-wing US outlets.

Each week, we would like to invite readers of Stats Chat to submit nominations for our Stat of the Week competition and be in with the chance to win an iTunes voucher.

Here’s how it works:

• Anyone may add a comment on this post to nominate their Stat of the Week candidate before midday Friday December 7 2012.
• Statistics can be bad, exemplary or fascinating.
• The statistic must be in the NZ media during the period of December 1 – 7 2012 inclusive.
• Quote the statistic, when and where it was published and tell us why it should be our Stat of the Week.

Next Monday at midday we’ll announce the winner of this week’s Stat of the Week competition, and start a new one.

(more…)

The Herald (quoting the Daily Mail) asks “Is chocolate the new Viagra?“.

No, it isn’t. (more…)