Prompt Caching

Automatic prefix prompt caching with 75% to 90% cost reductions. Cache mechanics, hit reporting, and pricing benefits.

Prompt caching allows large shared context prefixes—such as documentation corpuses, codebases, API specifications, and detailed system prompts—to be reused across requests at a fraction of standard input pricing.

How Caching Works

Toknix handles prompt caching automatically. Developers do not need to configure complex cache control headers manually:

  • Prefix Matching: When a request shares an identical initial token sequence of 1,024 tokens or more with a recent request, the gateway routes to a warm cached instance.
  • Cache Lifetime: Cached tokens persist on high-speed memory for 5 minutes, refreshed automatically on every hit.
  • Cache Writes: Initial cache writes are charged at the standard input rate. Subsequent reads receive the discounted cached rate.

Cache Rates & Savings

Cache reads are heavily discounted across all supported models:

ModelStandard Input / 1MCached Read / 1MDiscount
GLM 5.3 Flash$0.03$0.00390% Off
Kimi K3$0.17$0.0476% Off
DeepSeek V4.1 Flash$0.04$0.005486% Off
Claude Opus 5.5$1.40$0.1490% Off
Gemini 4 Carbon$1.00$0.1090% Off

Verifying Cache Hits

Inspect the usage object in any completion response to audit cached tokens:

json
{
  "usage": {
    "prompt_tokens": 12500,
    "completion_tokens": 420,
    "total_tokens": 12920,
    "prompt_tokens_details": {
      "cached_tokens": 12000
    }
  }
}

In the example above, 12,000 tokens were billed at the cached rate, and only 500 tokens were billed at the standard rate.