Kōkako goneburger?

The North Island kōkako is one of Aotearoa’s most elegantly beautiful birds, and while rare, they still exist in the wild as well as in sanctuaries. I’ve seen them on Tiritiri, near Auckland.  The South Island kōkako is a bit more controversial. It has been regarded as extinct and but is officially classified at the moment as “Don’t Know”.

This week, the Press published a story about the South Island kōkako, based on a publication in a regional ornithology journal, claiming there was a 48% chance that the species is still around.  The story raises two questions: what even does that mean, and is it reasonable?

We know that many of the reported sightings of South Island kōkako must be wrong: if they were not, the bird would be everywhere and there would be no uncertainty about its survival.  The question is whether any of the reported sightings are correct. Now, if there are no South Island kōkako then clearly all the sightings are mistaken, no matter how skilled and careful the reporters are– just like all the sightings of Bigfoot.  If South Island kōkako are rare, then most of the sightings are mistaken, but some of them are probably correct. Most of the sightings aren’t all that convincing anyway, but some of them do look pretty convincing.

There are various ways to approach this problem statistically. One is to try to pick out some sightings that you are sure are correct and see whether these stop at some point, or become less frequent. Another is to look at sightings during the period we know the bird existed and see what the ratio of convincing to dodgy sightings was like then, and see if it changed.  These (described more elegantly and formalised with maths) are the methods of three papers by Andrew Solow and co-workers (a,b,c — the copyright industry probably won’t let you read them).

One of the key bits of data in this calculation is a 2007 observation of the South Island kōkako that the Ornithological Society of NZ thought was convincing. According to the research paper, the last reliable sightings from the period when  the bird was uncontroversially still around were five over the period 1954-1967.  The 48% kōkako probability in the new report relies very heavily on the bird not being extinct in 2007. Without that one report, the estimated survival probability would be basically zero.  The isolated 2007 sighting, if true, would also provide evidence that real sightings are rare even when the species is still present.

There’s a problem with the formulation of the extinction models.  The original paper describing the first method, the one that gives the 48% probability, says “The methods described in this note assume that, prior to extinction, sightings follow a stationary Poisson process”.  In English: we assume that (true) sightings occur independently at a constant underlying rate.  They probably don’t.  There are a lot more people out there now than in 1967, so the rate is probably not constant. Also, there will be clustering: if someone convincingly reports seeing a South Island kōkako, the birding community will descend on the area with cameras at the ready and the chance of true sightings should go up[1]  And if the population is diminishing slowly (as it would have to be), the true sightings will also diminish slowly. This method also requires that you can tell which sightings are true.

The third method I linked above allows for uncertain sightings, so you don’t have to be able to tell in advance which sightings are true. However, to make the maths tractable, it still models both true and false sightings as being stationary Poisson processes: there’s a constant random rate of true sightings before extinction and a constant random rate of false sightings before and after extinction. Under this model, if the kōkako is extinct then at least 99.75% of the sightings since 1967 are false.

That’s less impressive than it sounds. To start with, obviously if there aren’t any kōkako now and there were no reliable sightings between 1967 and 2007, then nearly all sightings are false.  Also, this doesn’t mean that people’s accuracy in distinguishing kōkako from other things is less than 1 in 100. The iNaturalist site records 350,000 observations with photo or sound recording of securely-identified birds that aren’t South Island kōkako over just the time since 2012, and people may have seen birds and not posted about it to iNaturalist. The proportion of times someone sees something and wrongly think it’s a South Island kōkako could still be tiny — it’s just large compared to the (possibly zero) number of true sightings.

So, overall the paper says that if there were South Island kōkako in 2007 it’s not unreasonable that there still are a few. Which is fair. If they exist, they’re probably in one remote area rather than all over the South Island.   The 48% probability was correctly presented in the research paper as the output of the statistical method they used, but you shouldn’t put a lot of weight on the precise number. When you don’t have good data to put into the model you aren’t going to get much certainty out of it, and the statistical modelling had to make some pretty big approximations.  In particular, the model is leaning quite hard on the approximation that the search effort (and number of false sightings) has not increased over time.

[1] there’s a type of statistical model called a “self-exciting point process”, whose name is very appropriate here.