Glossary term
Glossary term
Foundations
A single update of a model's parameters—the model's weights and biases—during training. The batch size determines how many examples the model processes in a single iteration. For instance, if the batch size is 20, then the model processes 20 examples before adjusting the parameters.
When training a neural network, a single iteration involves the following two passes:
A forward pass to evaluate loss on a single batch.
A backward pass (backpropagation) to adjust the model's parameters based on the loss and the learning rate.
See Gradient descent in Machine Learning Crash Course for more information.
J
Created for this library
An ML platform team tracks training iterations as a fundamental progress unit alongside epoch and step counts.
A research team logs metrics at every iteration during early experimentation to spot instabilities quickly.
A startup engineer monitors loss every few hundred iterations during a long training run to confirm convergence is on track.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License