You’ve got to be carefully taught
From the Guardian
The first international beauty contest judged by “machines” was supposed to use objective factors such as facial symmetry and wrinkles to identify the most attractive contestants. … But when the results came in, the creators were dismayed to see that there was a glaring factor linking the winners: the robots did not like people with dark skin.
The way statistical `supervised learning’ algorithms work is that you give the algorithm a lot of measurements, and a training set of ‘correctly’ classified examples. It tries to find the best way to get the ‘correct’ answers from the measurements, in an objective, unprejudiced way.
The algorithm doesn’t know or care about race. There’s nothing in the maths about skin colour or eye shape. But if it never saw a ‘beautiful’ label on anyone who looked like Zoë Kravitz, it’s not going to spontaneously generalise its idea of beauty.
We’re lucky people learn in a much more sophisticated way than computers, aren’t we?
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »