Tokens and Billing / TOKEN WIKI
context cache
In one sentence
Context caching is a mechanism that reuses processed context when specific conditions are met to reduce repeated calculations.
Why it matters
Workloads with many duplicate prefixes may benefit, but caching rules, validity period, and unit pricing vary by platform.
Practical considerations
Use real requests to verify the hit rate, and record hit and miss usage respectively; do not assume that all similar requests can hit. Will
FAQ
be cheaper if cache is turned on?
Not necessarily, it depends on the hit rate, cache storage or write charges and actual billing rules.
