Glossary term
Glossary term
Foundations
An LLM's tendency to use information from the start and end of a long context window more effectively than information from the middle. That is, given a long context, the lost-in-the-middle effect causes accuracy to be:
Relatively high when the relevant information to form a response is near the beginning or end of the context.
Relatively low when the relevant information to form a response is in the middle of the context.
The term comes from Lost in the Middle: How Language Models Use Long Contexts.
Created for this library
A long-context QA team mitigates the lost-in-the-middle effect by re-ranking retrieved passages so the most relevant appear at the start of the context.
A research team evaluates the lost-in-the-middle effect by measuring accuracy as a function of the position of the relevant passage in the prompt.
An LLM platform team designs its prompts so critical context is at the start or end, mitigating the lost-in-the-middle effect.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License