Glossary term
Glossary term
Training and Fine-Tuning
A family of techniques for converting an unsupervised machine learning problem into a supervised machine learning problem by creating surrogate labels from unlabeled examples.
Some Transformer-based models such as BERT use self-supervised learning.
Self-supervised training is a semi-supervised learning approach.
Created for this library
An NLP team uses self-supervised learning to pretrain a base encoder on unlabeled text before fine-tuning on classification tasks.
A speech recognition team uses self-supervised learning on unlabeled audio to pretrain a base model before fine-tuning on transcribed data.
A computer vision team uses self-supervised learning on unlabeled images to pretrain a backbone before downstream fine-tuning.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License