The Immigration NZ model: recap
To begin with: Yes, everyone being evaluated was already eligible for deportation.
There were two main categories of feedback on this point: the ‘Manus Island’ tendency, arguing they’re all guilty and so it doesn’t matter how you treat them, and the people pointing out that a model could perhaps make better decisions that an individual immigration officer. The first group have, I think, missed an important issue: the arguments given by Immigration NZ for this model being a good thing would apply anywhere else in the immigration system or the justice system where there is currently discretion — eg, police discretion to prosecute.
The second group do have a good point (which is why it’s a point I made in my original post), but only if the model is constructed well and, ideally, audited. As I said, it didn’t look like we had that sort of model. Today, we got more information about the model, thanks to Radio NZ’s Morning Report. Here’s a PDF of the spreadsheet and the briefing document (dated April 6, so potentially cleaned up after the initial publicity). It’s a spreadsheet, simply adding up points for a bunch of categories, with minimal scaling for importance based on Immigration NZ’s expert knowledge or fitting to empirical data.
It’s not especially surprising that the harm model is a bit crap. What is surprising is that the Minister thinks this is a good thing
He said he was concerned about misconceptions around the pilot programme.
“Some people were talking about a sophisticated algorithm some people were talking about racial profiling, both of those are incorrect and I think it’s very important that the public know exactly what this is, and what it isn’t,” he said.
“This is not modelling or a predictive tool – this is a spreadsheet that they put some information into and they rank people based on that information.”
That’s not a defence; it’s an indictment.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
Its a bit hard relying on Morning Report, as we have ‘evidence’ that their reporting is sloppy.
6 years ago
We’re not relying on Morning Report for the *content* of the documents: those are from the Minister, under the OIA.
6 years ago
“Here’s a PDF of the spreadsheet and the briefing document (dated April 6, so potentially cleaned up after the initial publicity).”
The ‘aide memoire’ was specifically written after the publicity , not before. So its seems unusual to accuse them of ‘cleaning up’
Profiling is often used in a medical research sense, as a quick look at Google scholar provides.
eg
“DNA profiling analysis of endometrial and ovarian cell lines reveals misidentification, redundancy and contamination.”
So there may be different tools and established methods of profiling for law enforcement purposes,these seem to be concentrated on racial profiling which this one didnt do as ethnicity or country of origin isnt included.
INZ just say that its a misunderstanding of the word profiling and its just the set of criteria on their spreadsheet. I would imagine the Police, IRD etc would have different methods and processes when they build their profiles.
6 years ago
In the initial report, Immigration NZ were unambiguous that they used country of origin. The spreadsheet they presented today doesn’t, so either the initial statement was wrong or the spreadsheet has been changed.
I do agree completely that ‘profiling’ isn’t a problem. As you say, it has lots of perfectly innocent uses. In fact, even in the phrase ‘racial profiling’, it’s ‘racial’ that’s problematic, not ‘profiling’.
6 years ago
I also think that it is critical to include the question of what created eligibility for deportation. It is quite possible to accidentally violate immigration laws in a country, especially if they are complex and not very opaque. It is one thing to depart dangerous criminals. It’s another to nab people because there are teething problems with the website to report changes in address.
This is where a model might be very useful by looking to see what different classes of violations predict for public safety.
That said, a spreadsheet like that doesn’t really help. And Excel is famous for being very mutable — who knows what happened to the country of origin column (never there, typical excel deletion). I am very suspicious of any statistical “model” using Excel.
6 years ago