Tag: prompt-caching
All the articles with the tag "prompt-caching".
Prompt Caching: Paying Once for a Prefix
Updated:API prompt caching is the KV cache and prefix sharing sold as a price. What is actually cached, why only an exact prefix can hit, breakpoints and lifetimes, the write and read prices and the break-even worked from published multipliers, the time a cached prefix saves measured on a 1.5B model, the gigabytes a provider holds for one, how to order a prompt, and how agents break their own cache.