Glossary term
Glossary term
Training and Fine-Tuning
A form of regularization useful in training neural networks. Dropout regularization removes a random selection of a fixed number of the units in a network layer for a single gradient step. The more units dropped out, the stronger the regularization. This is analogous to training the network to emulate an exponentially large ensemble of smaller networks. For full details, see Dropout: A Simple Way to Prevent Neural Networks from Overfitting.
Created for this library
A computer vision team adds dropout regularization in the dense head of its classifier to prevent overfitting on a small medical dataset.
An NLP team applies dropout regularization in transformer layers to keep generalization stable when fine-tuning on a small enterprise corpus.
A speech recognition team uses dropout regularization to keep its acoustic model robust across speakers and microphones.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License