What is prompt caching?

Prompt caching lets a provider store the beginning of a prompt after the first request and reuse it on later requests that start with the same text. Cached input tokens are billed at a lower rate than normal input tokens and are processed faster, which cuts cost and latency for long, repeated prompts.

Also called: Context caching, Cached input tokens · Last updated

When it helps

Caching pays off when many requests share a long prefix: a large system prompt, tool definitions, a document under discussion, or the earlier turns of a conversation. Put the stable content first and the changing content last so the shared prefix stays identical.

What it costs

Providers publish a separate cached-input price, and some also charge to write the cache. Model pages on LLM Gateway list the cached input price per provider next to the normal input price.

Learn more

Related terms