- Glossary
- Prompt caching
What is prompt caching?
Prompt caching lets a provider store the beginning of a prompt after the first request and reuse it on later requests that start with the same text. Cached input tokens are billed at a lower rate than normal input tokens and are processed faster, which cuts cost and latency for long, repeated prompts.
Also called: Context caching, Cached input tokens · Last updated
When it helps
Caching pays off when many requests share a long prefix: a large system prompt, tool definitions, a document under discussion, or the earlier turns of a conversation. Put the stable content first and the changing content last so the shared prefix stays identical.
What it costs
Providers publish a separate cached-input price, and some also charge to write the cache. Model pages on LLM Gateway list the cached input price per provider next to the normal input price.