Back to blog

How to Track LLM Usage and Spend with the API

An LLM cost tracking tutorial: read the exact USD cost of every request from the response's usage object, segment spend by user and feature with metadata headers, enforce hard budgets with per-key spending limits, and audit everything in the dashboard.

A glowing cost meter and coins on a circuit board, representing LLM usage and spend tracking

Most teams discover their LLM spend the same way: on the invoice. Provider dashboards lag, tokens don't map cleanly to dollars, and once several models and providers are involved, nobody can answer "what did feature X cost last week?"

LLM Gateway makes LLM cost tracking a property of every request. Each API response carries its exact USD cost, every request is logged with full detail, and budgets are enforced at the key level — so spend is something you read in code, not reconstruct at month-end.

Read the cost from every response

Every chat completion includes a usage object with real-time cost data:

1{2  "usage": {3    "prompt_tokens": 10,4    "completion_tokens": 15,5    "total_tokens": 25,6    "cost": 0.000125,7    "cost_details": {8      "input_cost": 0.000025,9      "output_cost": 0.0001,10      "cached_input_cost": 0,11      "request_cost": 0,12      "web_search_cost": 0,13      "data_storage_cost": 0.0000002514    }15  }16}

cost is the total inference cost in USD; cost_details splits it into input, output, cached-input, per-request, web-search, and storage components. Track it inline:

1import OpenAI from "openai";2
3const client = new OpenAI({4  apiKey: process.env.LLM_GATEWAY_API_KEY,5  baseURL: "https://api.llmgateway.io/v1",6});7
8const response = await client.chat.completions.create({9  model: "gpt-4o",10  messages: [{ role: "user", content: "Hello!" }],11});12
13const usage = response.usage as { cost?: number };14console.log(`Request cost: $${usage.cost?.toFixed(6)}`);

Streaming responses include the same fields in the final usage chunk before [DONE], so streamed traffic is just as accountable. The cost breakdown docs document every field, including cache-write premiums and image and audio cost components.

Enforce a budget in code

Because cost arrives with the response, budget enforcement is a few lines:

1let totalSpent = 0;2const BUDGET_LIMIT = 10.0; // $103
4async function makeRequest(messages: Message[]) {5  const response = await client.chat.completions.create({6    model: "gpt-4o",7    messages,8  });9
10  totalSpent += (response.usage as { cost?: number }).cost ?? 0;11  if (totalSpent > BUDGET_LIMIT) {12    throw new Error(`Budget exceeded: $${totalSpent.toFixed(2)}`);13  }14  return response;15}

Segment spend by user, tenant, or feature

Attach metadata to any request with X-LLMGateway-* headers:

1curl -X POST https://api.llmgateway.io/v1/chat/completions \2  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \3  -H "Content-Type: application/json" \4  -H "X-LLMGateway-Tenant-ID: acme-corp" \5  -H "X-LLMGateway-User-ID: user-12345" \6  -H "X-LLMGateway-Feature: chat-assistant" \7  -d '{ "model": "gpt-4o", "messages": [{ "role": "user", "content": "Hi" }] }'

Requests are logged with their metadata, and the Activity view can filter by any custom header value — which turns "what did the chat-assistant feature cost?" into a filter instead of a data-engineering ticket. See the metadata docs.

Put hard caps where the risk is

Per-request tracking tells you what happened; per-key limits stop what shouldn't. Each API key supports two independent spending limits:

  • All-time limit — a lifetime cap for the key
  • Recurring limit — like $10/day or $500/month, resetting automatically

When a key hits either limit, requests return 401 until the limit resets or is raised. Give every service, agent, and environment its own key and a runaway loop becomes a bounded incident, not an invoice.

Audit it all in the dashboard

Everything above also lands in the dashboard, per project and per organization:

Costs are split between credits and bring-your-own provider keys, and image, video, and audio generations carry their own cost components — so multimodal workloads stay as accountable as text.

Frequently Asked Questions

Does cost tracking work with streaming?

Yes. The final usage chunk of a streamed response includes cost and cost_details, identical to non-streaming responses.

Does this work if I bring my own provider keys?

Yes. Requests routed through your own provider keys are logged and costed the same way — the dashboard splits spend between credits and BYOK traffic.

Can I track image, video, and audio costs too?

Yes. The usage schema includes image input/output, audio, and video cost fields, and the dashboards break modality costs out — see the guides on image, video, and audio generation.

Is cost tracking gated to a paid plan?

No. Cost breakdown in API responses is available to all users, on both hosted and self-hosted deployments.

Get started: