- Glossary
- LLM routing
What is LLM routing?
LLM routing is the choice of which provider, and sometimes which model, handles each request. A router weighs price, latency, uptime, and the features a request needs, then sends it to the best available option. Routing lets one application use many providers without hard-coding any of them.
Also called: Model routing, Provider routing · Last updated
Model routing and provider routing
Provider routing picks among the providers that serve the model you asked for. Model routing goes further and picks the model itself based on the request. Both aim for the same result: an acceptable answer at the lowest cost and delay.
Routing on LLM Gateway
Request a model by its ID and the gateway selects a provider that can handle your prompt size and parameters. Prefix the ID with a provider, such as openai/gpt-4o, to pin one.