Glossary term
Glossary term
Training and Fine-Tuning
A loss function—used in conjunction with a neural network model's main loss function—that helps accelerate training during the early iterations when weights are randomly initialized.
Auxiliary loss functions push effective gradients to the earlier layers. This facilitates convergence during training by combating the vanishing gradient problem.
Created for this library
A multi-task vision team adds an auxiliary segmentation loss when training a detection model so the encoder learns sharper boundary features.
A speech recognition vendor adds a phoneme-level auxiliary loss to its end-to-end model to encourage the encoder to learn linguistically meaningful units.
A search ranking team trains its main model with an auxiliary click-prediction loss so the shared encoder also produces useful intermediate representations.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License