Glossary term
Glossary term
Governance and Compliance
A systematic tendency in data, design, model behavior, or deployment that can produce unfair, inaccurate, or harmful outcomes for certain individuals or groups. Bias assessment should consider historical bias, sampling bias, measurement bias, label bias, deployment bias, and feedback loops after release.
Buolamwini and Gebru's Gender Shades study (2018) documented disproportionate error rates of commercial facial analysis systems on darker-skinned women.
Obermeyer et al. (2019) found a widely used US healthcare risk algorithm assigned the same level of risk to Black patients as White patients despite higher illness in Black patients.
NIST SP 1270 (2022) Towards a Standard for Identifying and Managing Bias in AI categorizes bias types and provides governance guidance.