Glossary term
Glossary term
Foundations
A mathematical technique to minimize loss. Gradient descent iteratively adjusts weights and biases, gradually finding the best combination to minimize loss.
Gradient descent is older—much, much older—than machine learning.
See the Linear regression: Gradient descent in Machine Learning Crash Course for more information.
Created for this library
An ML team uses gradient descent with momentum as the default optimizer for production training pipelines.
A research team experiments with several gradient descent variants like Adam and Adafactor on its largest LLM to balance memory and speed.
An ML platform team standardizes on a tuned mini-batch gradient descent schedule per model family so engineers can focus on data and features.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License