Dynamic Routes
Routing logic that lives in the gateway, not your code
Hardcoding a model turns every routing decision into a code change: a different model for paid users, a cheaper one for simple tasks, a 10% canary for a new release. Dynamic Routes move that logic into the gateway. A route is a decision graph of conditions on request headers, body fields, or gateway metadata, classifier branches on difficulty, task, or output type, percentage splits, and model targets, and your application calls it by setting the model to dynamic/<name>. Build it in the visual drag-and-drop editor or as JSON. Edits go to a draft; publishing snapshots an immutable, numbered version, and rolling back re-points the route at any earlier version instantly. Once a route resolves a model, the request goes through the same smart routing as any other: provider scoring, sticky sessions, and cross-provider fallback all still apply. For project-wide defaults, see Per-Project Routing Overrides.
Why teams turn it on
Re-route without a deploy
Swap a model, add a branch, or shift a split in the dashboard. Your application keeps calling the same dynamic/<name>.
Conditions and classifiers
Branch on a header like x-user-tier, a body field like max_tokens, org, project, or key metadata, or a classifier verdict on difficulty, task, or output type.
A/B splits and canaries
Percentage nodes divide traffic between targets, so a new model can take 10% before it takes everything.
Versioned, with instant rollback
Publishing creates an immutable version. Rolling back re-points the route, with no data migration and no redeploy.
How it works
From decision to deployed in three short steps
- 01
Build the graph
Under Project Settings → Dynamic Routes, combine conditional, classifier, percentage, model, and end nodes in the visual editor or as JSON.
- 02
Publish a version
Publishing snapshots the draft as an immutable, numbered version. A route serves traffic only when it is enabled and published.
- 03
Call it by name
Set the model to dynamic/<name>. Each request records the route, version, and node path in its routing metadata.
bash
Call a dynamic route
curl https://api.llmgateway.io/v1/chat/completions \
-H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
-H "x-user-tier: paid" \
-d '{
"model": "dynamic/support",
"messages": [{"role": "user", "content": "Hello!"}]
}'Real-world use cases
Why customers actually adopt this
Tiered quality
Send paid users to a frontier model and free users to a fast, cheaper one, keyed on a request header.
Cost-aware routing by task
Let the classifier send simple extraction to a small model and hard analysis to a large one.
Safe model migrations
Canary a new model on a slice of traffic, check the route, version, and node path of each request in the activity log, then publish the full switch.
Frequently asked
- Which endpoints resolve dynamic routes?
- /v1/chat/completions and the endpoints that proxy through it: /v1/responses, /v1/messages, and the AI SDK gateway endpoints. Embeddings, images, audio, video, rerank, OCR, and moderations do not.
- How does a route finish?
- Evaluation starts at the entry node and follows branches until it reaches a model node, which routes the request, or an end node, which rejects it with a 400.
- Does provider fallback still apply?
- Yes. Once a route resolves a model, provider scoring, sticky sessions, and automatic cross-provider fallback apply as usual. Published routes are also cached with a stale fallback, so recently used routes keep resolving if the database is briefly unreachable.
More enterprise capabilities
The rest of the enterprise stack
Organization-Wide Analytics
Cost, requests, and tokens totaled across every project, with breakdowns by model, project, API key, and member. Built on pre-aggregated rollups, so any date range stays fast.
Per-Member Budgets & Developer Role
Per-member spend caps enforced at request time, org-wide default developer limits, and a project-scoped Developer role. An over-budget request is rejected before it ever reaches a provider.
Teams & Directory Sync
Group developers under a shared project ceiling, per-developer budgets, and IAM rules. Microsoft Entra groups map onto teams over SCIM.
Enterprise Audit Logs
Tamper-evident audit trails for SOC 2, HIPAA, ISO 27001, and internal investigations. Every config change, key rotation, and admin action — captured, attributed, exportable.
Data Streams to Your SIEM
Forward audit logs to your SIEM and export request log metadata to your own HTTPS endpoint in signed, ordered batches, with replay of up to 30 days.
Per-Project Routing Overrides
Override global routing rules at the project level — region, provider order, fallback chain, and cost ceilings. Production stays pinned; experimental teams stay flexible.
Enterprise Guardrails
Server-side detection for prompt injection, PII, secrets, and policy violations. Configured centrally, enforced at the gateway, auditable per-request.
Provider Compliance Policies
Require SOC 2, ISO 27001, GDPR, zero data retention, no prompt training, or approved headquarters countries, and the gateway refuses every provider that falls short.
Zero Data Retention
Enforce zero data retention across provider routing, gateway storage, response caches, and the Responses API, for the whole organization or a single API key.
Discord & Slack Alerts
Native webhook integrations for Discord and Slack. Get the enterprise contact-sales form, billing events, guardrail trips, and SLA breaches in the channels your team already monitors.
Single Sign-On (SAML / OIDC)
SAML 2.0 and OIDC SSO with SCIM provisioning, group-based role mapping, and enforced-only access. No local credentials, no shared passkeys, no off-boarding gaps.
Master Keys & Provisioning API
Org-scoped bearer tokens that create projects, gateway API keys, IAM rules, and custom providers from your backend, and report cost per member and model.
Organization Skills
Publish shared agent skills once and let developers install them into Claude Code, Codex, Cursor, OpenCode, and other agents with one CLI command.
White-Label Chat & Playground
Embed or stand up a fully white-labeled chat app and playground under your own domain. Customize branding, default models, system prompts, and feature toggles.
See dynamic routes on your real workloads
Bring a sample workload to a 30-minute call. We'll wire it up live and show you the actual experience your team will get.