Knowing what you’re predicting

From a Sydney Morning Herald story about brain wave reading.

The faux insurgents were asked to hatch a mock terrorist plot by selecting one of four dates in July, one of four locations in Houston and one of four types of bomb, then jot it all down in a letter to their terrorist boss.

EEG caps on, they were later shown a slew of months of the year, US cities and varieties of terror attack on a computer; and when “July”, “Houston” and “bomb” appeared among them, the P300 spikes were big enough to nab all 12 “culprits”.

The brain fingerprinting technique relies on picking up a signal that the brain recognises some piece of information. The people who make the gadgetry claim this can be done with 100% accuracy (not everyone agrees). However, even if the brain waves can be picked up with 100% accuracy, that’s not 100% accuracy for the real question.

Consider DNA evidence. In the ideal case of a high-quality DNA sample from the scene of a crime, and a high-quality sample from a suspect, and the right combination of ancestries, it is possible to be almost 100% sure that the suspect’s DNA (or that of an identical twin) is present in the crime sample. The scene-of-crime sample could be billions of times more likely if the suspect contributed to it than if a random person from the population did. The DNA expert won’t (or shouldn’t) testify that the suspect is almost certainly guilty, because that’s not a DNA question. Even ruling out police fraud or incompetence, the suspect’s DNA could have present in the sample for some innocent reason. Guilt is not a question that capillary electrophoresis can answer.

The situation is worse for the brain fingerprinting technique, because it’s intended to be used before a terrorist attack has been committed, and potentially before the suspects have even committed a crime such as conspiracy.  Maybe they recognised an attack plan because they’d been thinking about it, or because they’d read a Tom Clancy novel about it. Maybe they recognised “July” and “Houston” from baseball and the bomb from somewhere else entirely.  None of these would be counted as an error by the brain wave enthusiasts — they are entirely genuine indications of recognition — but they aren’t specific evidence of past or future crime.