Glossary term
Glossary term
Infrastructure and Serving
A Model Router decides which AI model to use for a specific task. Based on factors like prompt type, confidence score, or domain, it directs requests to the best-fitting model, ensuring the system stays efficient, accurate, and scalable.
The algorithm that determines the ideal model for inference in model cascading. A model router is itself typically a machine learning model that gradually learns how to pick the best model for a given input. However, a model router could sometimes be a simpler, non-machine learning algorithm.
OpenRouter routes API calls across over 300 models from OpenAI, Anthropic, Google, and Meta.
Martian and NotDiamond are AI-powered model routers that pick the best model per prompt.
LangChain's RouterChain and Portkey's gateway support model routing in production agent systems.
Created for this library
A SaaS team uses a model router to send simple prompts to a Flash model and reserve a larger model for complex prompts.
An LLM platform team builds a model router that selects per-tenant routing rules so each customer can balance cost and quality.
An enterprise ML platform uses a model router that picks the cheapest model meeting the quality threshold for each request.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License