Gender Shades

Summary

Joy Buolamwini describes how facial-analysis systems failed to detect her face or classified her as male. This prompted her to investigate whether these problems affected other people. For her MIT thesis project, Gender Shades, she evaluated commercial gender-classification systems for differences in performance across gender and skin tone.

Method

Buolamwini created a dataset of more than 1,000 images of parliament members from three African and three European countries, aiming to represent a range of skin tones. She tested systems from IBM, Microsoft, and Face++.

Findings

• Overall accuracy: Microsoft had the highest overall accuracy, at 94%. All three systems performed better on male faces than on female faces, and better on lighter-skinned subjects than on darker-skinned subjects.
• Subgroup performance: Accuracy was lowest for darker-skinned women. IBM and Microsoft performed best on lighter-skinned men, while Face++ performed best on darker-skinned men relative to the other systems. IBM showed the largest disparity: a 34-percentage-point difference in error rates between lighter-skinned men and darker-skinned women. For darker-skinned women, correct classification rates approached a coin toss.

Limitations and concerns

The systems used only “male” and “female” labels and did not distinguish gender identity from biological sex. Buolamwini notes that limited diversity in training images and benchmark datasets may contribute to unequal accuracy. Reporting results only in aggregate can also conceal disparities.

Broader implications

Similar machine-learning methods are used in facial recognition and other AI applications, so biased performance can have consequences beyond gender classification. Buolamwini calls for greater transparency and accountability in commercial AI, warning that automated systems can reproduce inequality rather than operate neutrally.

Main speaker/source: Joy Buolamwini, discussing the Gender Shades project and its findings.