Glossary term
Glossary term
Memory and Retrieval
Contextual Embedding transforms words or data into numerical vectors that capture meaning based on surrounding context, helping AI distinguish nuances like bank as a financial institution versus a riverbank for smarter retrieval and reasoning.
BERT introduced contextual embeddings in 2018, transforming search and NLP across Google products.
OpenAI text-embedding-3-large and Cohere Embed v3 are widely used contextual embedding models.
Anthropic's Contextual Retrieval method, published in 2024, enriches each chunk with surrounding context before embedding.