Glossary term
Glossary term
Foundations
A human who provides labels for examples. "Annotator" is another name for rater.
See Categorical data: Common issues in Machine Learning Crash Course for more information.
Created for this library
A search-quality team employs raters to score query-document pairs as relevant or not relevant for training and evaluation.
An LLM evaluation team employs raters to score generated responses against a rubric so model release decisions are grounded in human judgment.
A medical AI team uses radiologists as raters to label tumor presence on chest X-rays for training ground truth.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License