TanStack AI + LLM Gateway: One Adapter, 200+ Models

Every provider adapter in your chat app hardwires a vendor. Build on TanStack AI with the OpenAI adapter and trying Claude means a new package, a new API key, new billing, and a code change. Multiply that by every model your team wants to evaluate, and "let's compare models" becomes a sprint instead of an afternoon.

That friction is now gone: TanStack AI ships a first-party LLM Gateway adapter, merged into the TanStack AI repository alongside the OpenAI and Anthropic adapters. Install @tanstack/ai-llmgateway, set one API key, and your app reaches 200+ models from 40+ providers — switching between them is a one-line string change.

What is TanStack AI?

TanStack AI is the headless AI framework from the team behind TanStack Query, Router, and Table. The server side is a typed chat() primitive with pluggable provider adapters; the client side is a useChat hook for React, Vue, Svelte, Angular, and Preact; and the two talk over the AG-UI event protocol, so streaming text, tool calls, and reasoning all arrive as structured events instead of a raw text stream.

The adapter is the seam where the provider plugs in. That is exactly where a gateway belongs: one adapter that speaks to every provider, instead of one adapter per provider.

Install the TanStack AI LLM Gateway adapter

1pnpm add @tanstack/ai @tanstack/ai-react @tanstack/ai-llmgateway

1pnpm add @tanstack/ai @tanstack/ai-react @tanstack/ai-llmgateway

Create an API key in the LLM Gateway dashboard and export it:

1export LLM_GATEWAY_API_KEY=llmgtwy_your_key_here

1export LLM_GATEWAY_API_KEY=llmgtwy_your_key_here

One route, one hook, streaming chat

The server route creates a stream with chat() and returns it as server-sent events. llmGatewayText reads your key from the environment:

1// app/api/chat/route.ts2import { chat, toServerSentEventsResponse } from "@tanstack/ai";3import { llmGatewayText } from "@tanstack/ai-llmgateway";4
5export async function POST(request: Request) {6  const { messages } = await request.json();7
8  const stream = chat({9    adapter: llmGatewayText("gpt-5.6-terra"),10    messages,11  });12
13  return toServerSentEventsResponse(stream);14}

1// app/api/chat/route.ts2import { chat, toServerSentEventsResponse } from "@tanstack/ai";3import { llmGatewayText } from "@tanstack/ai-llmgateway";4
5export async function POST(request: Request) {6  const { messages } = await request.json();7
8  const stream = chat({9    adapter: llmGatewayText("gpt-5.6-terra"),10    messages,11  });12
13  return toServerSentEventsResponse(stream);14}

The client connects useChat to that route — no API key in the browser, no provider-specific wiring:

1// components/chat.tsx2"use client";3
4import { fetchServerSentEvents, useChat } from "@tanstack/ai-react";5import { useState } from "react";6
7export function Chat() {8  const [input, setInput] = useState("");9  const { messages, sendMessage, isLoading } = useChat({10    connection: fetchServerSentEvents("/api/chat"),11  });12
13  return (14    <div>15      {messages.map((message) => (16        <div key={message.id}>17          <strong>{message.role === "assistant" ? "Assistant" : "You"}</strong>18          {message.parts.map((part, index) =>19            part.type === "text" ? <p key={index}>{part.content}</p> : null,20          )}21        </div>22      ))}23      <form24        onSubmit={(event) => {25          event.preventDefault();26          if (!input.trim() || isLoading) {27            return;28          }29          sendMessage(input);30          setInput("");31        }}32      >33        <input34          value={input}35          onChange={(event) => setInput(event.target.value)}36          placeholder="Say something..."37        />38      </form>39    </div>40  );41}

1// components/chat.tsx2"use client";3
4import { fetchServerSentEvents, useChat } from "@tanstack/ai-react";5import { useState } from "react";6
7export function Chat() {8  const [input, setInput] = useState("");9  const { messages, sendMessage, isLoading } = useChat({10    connection: fetchServerSentEvents("/api/chat"),11  });12
13  return (14    <div>15      {messages.map((message) => (16        <div key={message.id}>17          <strong>{message.role === "assistant" ? "Assistant" : "You"}</strong>18          {message.parts.map((part, index) =>19            part.type === "text" ? <p key={index}>{part.content}</p> : null,20          )}21        </div>22      ))}23      <form24        onSubmit={(event) => {25          event.preventDefault();26          if (!input.trim() || isLoading) {27            return;28          }29          sendMessage(input);30          setInput("");31        }}32      >33        <input34          value={input}35          onChange={(event) => setInput(event.target.value)}36          placeholder="Say something..."37        />38      </form>39    </div>40  );41}

The example uses a Next.js route handler; a TanStack Start server route works the same way — see the quick start for that variant.

Switch models without touching your UI

The model is a string, and LLM Gateway accepts it in two formats:

• Canonical IDs (gpt-5.6-terra, claude-sonnet-5) — the gateway routes to the best available provider based on uptime, throughput, price, and latency
• Provider-prefixed IDs (moonshot/kimi-k3) — pin a specific provider, with automatic failover if its uptime drops below 90%

1adapter: llmGatewayText("claude-sonnet-5"),  // was "gpt-5.6-terra" — that's the whole migration

1adapter: llmGatewayText("claude-sonnet-5"),  // was "gpt-5.6-terra" — that's the whole migration

A curated set of flagship models additionally carries typed metadata with editor autocomplete; every other ID on the models page still works. Your useChat component doesn't change either way, because the AG-UI events it consumes are provider-agnostic.

Tool calling works unchanged

Tools are defined once with toolDefinition and a Standard Schema (Zod works out of the box). TanStack AI runs the tool loop on the server, and the gateway forwards the calls to whichever provider is serving the model:

1import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai";2import { llmGatewayText } from "@tanstack/ai-llmgateway";3import { z } from "zod";4
5const getWeather = toolDefinition({6  name: "get_weather",7  description: "Get the current weather for a location",8  inputSchema: z.object({9    location: z.string(),10  }),11}).server(async ({ location }) => {12  return { temperature: 72, condition: "sunny" };13});14
15const stream = chat({16  adapter: llmGatewayText("gpt-5.6-terra"),17  messages,18  tools: [getWeather],19});

1import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai";2import { llmGatewayText } from "@tanstack/ai-llmgateway";3import { z } from "zod";4
5const getWeather = toolDefinition({6  name: "get_weather",7  description: "Get the current weather for a location",8  inputSchema: z.object({9    location: z.string(),10  }),11}).server(async ({ location }) => {12  return { temperature: 72, condition: "sunny" };13});14
15const stream = chat({16  adapter: llmGatewayText("gpt-5.6-terra"),17  messages,18  tools: [getWeather],19});

Reasoning models stream their thinking

Reasoning models stream reasoning_content deltas through the gateway, and the adapter surfaces them as AG-UI REASONING_* events — in useChat they arrive as thinking parts you can render or hide. Control the depth with reasoning_effort:

1const stream = chat({2  adapter: llmGatewayText("kimi-k3"),3  messages,4  modelOptions: {5    reasoning_effort: "high",6  },7});

1const stream = chat({2  adapter: llmGatewayText("kimi-k3"),3  messages,4  modelOptions: {5    reasoning_effort: "high",6  },7});

reasoning_effort accepts the extended scale none / minimal / low / medium / high / xhigh / max on top of OpenAI's standard tiers. Parameters a routed provider doesn't support are stripped server-side, so the same modelOptions stay portable across every model you try.

What the gateway adds underneath

The adapter is thin on purpose — the routing intelligence lives in the gateway:

• Automatic failover — if a provider goes down mid-launch-day, requests route to the next healthy one
• Per-request cost tracking — every TanStack AI request lands in your dashboard with tokens, cost, and latency
• Response caching — repeated requests are served from cache and cost nothing
• Budgets and limits — hard caps per organization, project, and API key, enforced at the gateway

None of that requires code in your TanStack AI app. It comes with the endpoint.

Self-host it if you need to

LLM Gateway is open source (AGPLv3). If requests must stay inside your own boundary, point the adapter at your deployment with createLLMGatewayText:

1import { createLLMGatewayText } from "@tanstack/ai-llmgateway";2
3const adapter = createLLMGatewayText(4  "gpt-5.6-terra",5  process.env.LLM_GATEWAY_API_KEY!,6  {7    baseURL: "https://gateway.internal.example.com/v1",8  },9);

1import { createLLMGatewayText } from "@tanstack/ai-llmgateway";2
3const adapter = createLLMGatewayText(4  "gpt-5.6-terra",5  process.env.LLM_GATEWAY_API_KEY!,6  {7    baseURL: "https://gateway.internal.example.com/v1",8  },9);

The adapter surface is identical against the hosted gateway at https://api.llmgateway.io/v1 and your own instance, so you can start hosted and move later without touching application code.

Getting started

• Try LLM Gateway free — create a key and point @tanstack/ai-llmgateway at it
• Read the TanStack AI docs page — full setup, tool calling, reasoning, and self-hosting reference
• Prefer the Vercel AI SDK? — the same one-key, every-model setup works there through @llmgateway/ai-sdk-provider