Glossary term
Glossary term
Architecture
A term used to describe a system that evaluates the text that both precedes and follows a target section of text. In contrast, a unidirectional system only evaluates the text that precedes a target section of text.
For example, consider a masked language model that must determine probabilities for the word or words representing the underline in the following question:
What is the _____ with you?
A unidirectional language model would have to base its probabilities only on the context provided by the words "What", "is", and "the". In contrast, a bidirectional language model could also gain context from "with" and "you", which might help the model generate better predictions.
For example, consider a masked language model that must determine probabilities for the word or words representing the underline in the following question:
What is the _____ with you?
Created for this library
A search team uses a bidirectional encoder on user queries so the model can use context from both sides of an ambiguous token.
A medical NLP team uses bidirectional models on clinical notes because the meaning of a term often depends on tokens that appear later in the sentence.
A customer support team uses a bidirectional model to classify ticket intents because the topic often becomes clear only at the end of the message.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License