February 2, 2017

Defining on-time arrival

Bernard Orsman, in the Herald, has written about Auckland bus punctuality, this time with data from Auckland Transport broken down by bus route.  The numbers look good overall, with, apparently, 96.36% of buses on time in January. If you caught a bus in January, you might find this surprising. The problem is that defining and then measuring the percentage of on-time buses is harder than it sounds.

The Auckland Transport number is the percentage of buses that depart their first stop within 5 minutes of schedule. That’s probably a good number for measuring whether the bus companies are delivering the service they’re being paid for. It’s not a good description of the lived experience of passengers.

At the other extreme, you could argue for a measurement averaged over all bus stops. That punctuality number would inevitably be lower, because of variation in traffic and traffic lights from trip to trip.  This isn’t the ideal measure in many ways, because the way to optimise it would be to have lots of slack in the bus timetable and force the bus to wait at every stop. But people do care about it. I bet Aaron Schiff that 80% of buses were within 5 minutes of schedule averaged over all stops and all trips using the bus GPS data. I’ve conceded: I think the true figure is probably more like 70%.

Another approach would be to  look at on-time performance at the timepoints on the official timetable for the route. For example, the 324, singled out in the Herald story, has Mangere Town Centre, Ōtāhuhu station, Ōtāhuhu Town Centre, and Seaside Park.  If you wanted an official benchmark statistic, that would be a reasonable choice. You’d expect to get a higher number than the all-trips/all-stops figure, but lower than the first-stop-only figure.

There are other possibilities, though. For a frequent service what matters isn’t the timetable but the waiting time between buses. You’d prefer to have all the buses 10 minutes late rather than alternate ones 10 minutes late and on time. “Maintenance of Headway” is the the technical term (and the title of a humorous novel about bus timetables. No, I’m not making this up).

Also, there can be more important things than adherence to a schedule.  On a rainy Friday evening the punctuality is going to be pretty bad, but your ability to get from point A to point B by bus is going to be better than on a typical Sunday morning.

The right choice of summary depends on what you’re trying to do: contractual audit, benchmarking for trends or against similar cities, describing what it typically feels like to passengers, or detecting that the system is having a bad time right now.  Personally, I’m most interested in the last of these: describing how performance varies over time with weather, school holidays, and other challenges, and how it varies over Auckland.

Whatever your aim, it’s important to have realistic expectations based on what summary you’re using: 90% punctuality would likely be unattainable taken over all stops, but it’s a bit average for just the first stop.


Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »