- Glossary
- Provider fallback
What is provider fallback?
Provider fallback is retrying a failed AI request on a different provider that serves the same model. When the first provider returns a server error, a rate limit, or a timeout, the request is sent to the next one inside the same API call, so the application receives an answer instead of an error.
Also called: LLM failover, Automatic failover · Last updated
What triggers it
Fallback applies to failures on the provider side: 5xx errors, timeouts, connection failures, and upstream rate limits. A request that is invalid fails the same way everywhere, so it is returned to the caller instead of retried.
When it does not run
On LLM Gateway, fallback is skipped when you pin a provider, send the X-No-Fallback: true header, or no other provider serves the model.