Realtime Voice API

Voice was the one thing you still had to route around the gateway. Realtime models speak a WebSocket event protocol instead of HTTP request/response, so every voice feature meant a second integration: provider keys in your app, no cost tracking, no usage limits, no fallback. The realtime API closes that gap — point any OpenAI realtime client at wss://api.llmgateway.io/v1/realtime and keep your existing event handling.

A drop-in WebSocket endpoint

Authenticate with your LLM Gateway API key in the Authorization header, exactly like the rest of the API:

1import WebSocket from "ws";2
3const ws = new WebSocket(4  "wss://api.llmgateway.io/v1/realtime?model=gpt-realtime",5  { headers: { Authorization: `Bearer ${process.env.LLM_GATEWAY_API_KEY}` } },6);7
8ws.on("open", () => {9  ws.send(10    JSON.stringify({11      type: "session.update",12      session: {13        type: "realtime",14        instructions: "You are a friendly assistant.",15        audio: { output: { voice: "marin" } },16      },17    }),18  );19});

1import WebSocket from "ws";2
3const ws = new WebSocket(4  "wss://api.llmgateway.io/v1/realtime?model=gpt-realtime",5  { headers: { Authorization: `Bearer ${process.env.LLM_GATEWAY_API_KEY}` } },6);7
8ws.on("open", () => {9  ws.send(10    JSON.stringify({11      type: "session.update",12      session: {13        type: "realtime",14        instructions: "You are a friendly assistant.",15        audio: { output: { voice: "marin" } },16      },17    }),18  );19});

The session speaks the standard event protocol — session.update, input_audio_buffer.append, response.create on the way in; session.created, response.output_audio.delta, response.done on the way out. Server-side voice activity detection, input audio transcription, and function calling all work as they do upstream. The model is locked at connection time, so a session.update that tries to swap session.model is rejected with a model_locked event rather than silently repricing the call. Browse the available realtime models and their per-token audio rates on the models page.

Browsers get ephemeral secrets, not your API key

Mint a short-lived client secret from your backend and hand that to the browser:

1curl -X POST "https://api.llmgateway.io/v1/realtime/client_secrets" \2  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "expires_after": { "anchor": "created_at", "seconds": 120 },6    "session": { "type": "realtime", "model": "gpt-realtime" }7  }'

1curl -X POST "https://api.llmgateway.io/v1/realtime/client_secrets" \2  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "expires_after": { "anchor": "created_at", "seconds": 120 },6    "session": { "type": "realtime", "model": "gpt-realtime" }7  }'

The response carries an ek_... secret the browser passes through the standard openai-insecure-api-key WebSocket subprotocol. Secrets live 10–300 seconds (60 by default), are minted for exactly one model, and can pin the input transcription model too. Every auth, credit, and model-access check runs at mint time and again at connect time. Credentials are never accepted as query parameters — a URL carrying token, api_key, or client_secret is rejected with 400 before the socket opens.

Audio billing that reconciles

Every response is metered when it completes, with text, cached, and audio tokens priced separately at the model's listed rates — no estimating from wall-clock time. Input audio transcription is billed against its own model, as its own line item, and the transcription model is pinned for the session on first use so a mid-call swap can't reprice work already committed. It all lands in your activity feed and analytics next to your regular requests, now with audio input and output cost broken out.

Because billing happens per response, the gateway's authorization gates run per response too: credits, API key status, usage limits, and IAM rules are checked before each generation — whether you triggered it with response.create or the model's own voice activity detection did. A blocked generation arrives as an error event on the session instead of running your balance negative.

Sessions carry safety limits by default, and are closed gracefully once in-flight responses are billed:

Talk to a model right now

Lounge ships a Voice Calls page at chat.llmgateway.io/realtime: pick a model and a voice, hit Start call, and talk. The live transcript builds as you speak, a voice activity indicator shows who has the floor, and every call is saved to your history with its duration, model, voice, and token usage so you can reopen the transcript later.

Realtime runs on pay-as-you-go organizations — your credits, or your own provider key when the project uses provider keys. WebSocket is the only transport for now; WebRTC, SIP, image input, and hosted tools are not supported yet.

Realtime API docs → | Try Voice Calls →