Glossary term
Glossary term
Evaluation and Benchmarks
Examples intentionally not used ("held out") during training. The validation dataset and test dataset are examples of holdout data. Holdout data helps evaluate your model's ability to generalize to data other than the data it was trained on. The loss on the holdout set provides a better estimate of the loss on an unseen dataset than does the loss on the training set.
Created for this library
An ML platform team requires holdout data for every production model release so reviewers see an honest estimate of model performance.
A risk modeling team locks holdout data per model release so audit trails are reproducible.
A retail forecasting team uses out-of-time holdout data to ensure model evaluation reflects future-like conditions, not past.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License