Glossary term
Glossary term
Training and Fine-Tuning
The vector of partial derivatives with respect to all of the independent variables. In machine learning, the gradient is the vector of partial derivatives of the model function. The gradient points in the direction of steepest ascent.
Created for this library
An ML engineer inspects gradient magnitudes during training to diagnose whether a model is making meaningful updates per step.
A research team logs gradient norms across layers to detect vanishing or exploding behavior early in long training runs.
An ML platform team enables gradient checkpointing in long-sequence models to trade some compute for memory during training.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License