Glossary term
Glossary term
Architecture
An autoregressive model is a machine learning technique that predicts the next value in a sequence by learning from the values that came before it. The core idea is simple: what happens next is influenced by what has already happened. By identifying patterns in historical data, the model can make informed predictions about what comes next.
OpenAI's GPT family from GPT-2 through GPT-5 are all autoregressive language models.
Meta's Llama 3 and Llama 4 are autoregressive transformer-based LLMs.
Anthropic's Claude models and Google's Gemini 1.5 and 2.5 follow autoregressive next-token prediction during generation.