Glossary term
Glossary term
Foundations
The sum of the following in a neural network:
the number of hidden layers
the number of output layers, which is typically 1
the number of any embedding layers
For example, a neural network with five hidden layers and one output layer has a depth of 6.
Notice that the input layer doesn't influence depth.
For example, a neural network with five hidden layers and one output layer has a depth of 6.
Notice that the input layer doesn't influence depth.
Created for this library
A computer vision team experiments with model depth from 18 to 152 layers on ResNet variants to find the best speed-accuracy trade-off for its mobile app.
An NLP team compares model depth versus width in transformer variants when training a domain-specific model under a fixed compute budget.
A startup's research team caps model depth on its on-device model to keep latency under a strict mobile budget.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License