Glossary term
Glossary term
Training and Fine-Tuning
A gradient descent algorithm that uses mini-batches. In other words, mini-batch stochastic gradient descent estimates the gradient based on a small subset of the training data. Regular stochastic gradient descent uses a mini-batch of size 1.
Created for this library
An ML team uses mini-batch SGD with momentum as the default optimizer for production training pipelines.
A research team uses mini-batch SGD with a cosine learning rate schedule as the baseline optimizer for ResNet-style training.
An ML platform team standardizes on tuned mini-batch SGD recipes per model family so engineers can focus on data and features.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License