- Glossary
- Time to first token (TTFT)
What is time to first token (TTFT)?
Time to first token (TTFT) is the delay between sending a request and receiving the first piece of the model's response. It measures how responsive a streaming model feels. TTFT depends on the model, the provider serving it, prompt length, and current load, and it is separate from how fast tokens arrive afterward.
Also called: TTFT, First-token latency · Last updated
TTFT versus throughput
TTFT tells you how long a user waits before anything appears. Throughput, in tokens per second, tells you how long the full answer takes. A chat interface needs a low TTFT; a batch job cares about total time.
Comparing providers
The same model can respond at different speeds on different providers. Model pages on LLM Gateway show measured performance per provider, so you can compare them before pinning one.