What is an AI gateway?

An AI gateway is a service that sits between your application and AI model providers. It exposes one API and one key, forwards each request to the right provider, and adds the controls a single provider does not: failover, cost tracking, rate limits, caching, and access policies across every model you use.

Also called: LLM gateway, LLM API gateway · Last updated

Why teams use one

Calling each provider directly means a separate SDK, key, bill, and rate limit for each. A gateway replaces that with one integration. Switching models becomes a change to the model field, and an outage at one provider no longer takes your feature down.

How LLM Gateway works

LLM Gateway exposes an OpenAI-compatible endpoint at https://api.llmgateway.io/v1. Change the base URL and API key in an OpenAI SDK and the same code reaches models from every supported provider. Requests that name only a model are retried on another provider when the first one fails.

Learn more

Related terms