Glossary term
Glossary term
Foundations
A state reached when loss values change very little or not at all with each iteration. For example, the following loss curve suggests convergence at around 700 iterations:
A model converges when additional training won't improve the model.
In deep learning, loss values sometimes stay constant or nearly so for many iterations before finally descending. During a long period of constant loss values, you may temporarily get a false sense of convergence.
See also early stopping.
See Model convergence and loss curves in Machine Learning Crash Course for more information.
Created for this library
An ML platform team flags training jobs that have not reached convergence by a configured step count for automatic review.
A computer vision team monitors the loss curve and stops training shortly after convergence to save compute without sacrificing quality.
A trading research team measures convergence on a validation set and rejects models that fail to converge within the assigned compute budget.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License