Glossary term
Glossary term
Security
Adversarial testing to find vulnerabilities and unsafe behavior in AI systems.
Anthropic's dedicated safety team conducts structured red teaming before every Claude release - using both human red-teamers and automated adversarial-prompt generators to probe for harmful-content bypasses.
Microsoft AI Red Team (AIRT) published findings from red-teaming Copilot products, including prompt injection via document content and data-exfiltration risks - using results to harden guardrails before GA release.
NIST AI RMF Playbook recommends red teaming as a core practice - a US bank hired an external red team to test their loan-underwriting LLM for demographic bias and found disparate-impact issues in 3 of 7 income bands.