Glossary term
Glossary term
Architecture
Transformer architecture using only the encoder stack with bidirectional self-attention, suited for understanding tasks.
BERT (Google, 2018) is the canonical encoder-only model - used by Google Search for query understanding, entity recognition, and passage relevance scoring across billions of daily queries.
RoBERTa (Facebook) fine-tuned on GLUE benchmarks powers many enterprise NLP pipelines for sentiment analysis, intent classification, and named entity recognition where generation is not needed.
DeBERTa (Microsoft) improves on BERT with disentangled attention and enhanced positional encodings - deployed in Azure AI Language for document classification, PII detection, and text analytics.