Glossary term
Glossary term
Agentic Systems
A training technique to improve the performance of LLMs. Fast decay involves rapidly decreasing the learning rate during training. This strategy helps prevent the model from overfitting to the training data, and improves generalization.
Created for this library
An ML platform team uses fast decay on the learning rate during the first epochs to find a stable region before slowing decay.
A speech recognition team uses fast decay early in training so the model quickly reaches a good basin before fine-tuning with a small learning rate.
A computer vision team uses fast decay on the learning rate after a large pretraining phase to anneal toward a robust solution.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License