Glossary term
Glossary term
Security
Use of an AI system outside approved or intended boundaries. Governance should consider both accidental misuse by legitimate users and deliberate misuse by malicious actors. Controls include approved-use boundaries, user training, technical restrictions, logging, abuse monitoring, and sanctions or escalation for misuse.
A New York attorney was sanctioned in Mata v. Avianca (June 2023) for filing ChatGPT-generated fabricated case citations, an example of misuse outside intended legal research boundaries.
The 2024 Hong Kong deepfake fraud where a finance worker transferred USD 25 million after a video call featuring deepfaked colleagues illustrates deliberate misuse of generative AI.
Multiple deepfake nude image incidents in US high schools in 2024 documented by The New York Times illustrate misuse of generative image models against students.