Glossary term
Glossary term
Foundations
In machine learning, a distinct unit within a hidden layer of a neural network. Each neuron performs the following two-step action:
Calculates the weighted sum of input values multiplied by their corresponding weights.
Passes the weighted sum as input to an activation function.
A neuron in the first hidden layer accepts inputs from the feature values in the input layer. A neuron in any hidden layer beyond the first accepts inputs from the neurons in the preceding hidden layer. For example, a neuron in the second hidden layer accepts inputs from the neurons in the first hidden layer.
The following illustration highlights two neurons and their inputs.
A neuron in a neural network mimics the behavior of neurons in brains and other parts of nervous systems.
Created for this library
An NLP research team probes individual neurons in a transformer to understand what linguistic features they activate on.
An ML platform team logs neuron-level activation statistics across layers to spot dead units during training.
A research team prunes neurons with persistently small magnitude weights to produce a smaller, faster production model.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License