- Glossary
- AI gateway
What is an AI gateway?
An AI gateway is a service that sits between your application and AI model providers. It exposes one API and one key, forwards each request to the right provider, and adds the controls a single provider does not: failover, cost tracking, rate limits, caching, and access policies across every model you use.
Also called: LLM gateway, LLM API gateway · Last updated
Why teams use one
Calling each provider directly means a separate SDK, key, bill, and rate limit for each. A gateway replaces that with one integration. Switching models becomes a change to the model field, and an outage at one provider no longer takes your feature down.
How LLM Gateway works
LLM Gateway exposes an OpenAI-compatible endpoint at https://api.llmgateway.io/v1. Change the base URL and API key in an OpenAI SDK and the same code reaches models from every supported provider. Requests that name only a model are retried on another provider when the first one fails.