Tag: context-engineering
All the articles with the tag "context-engineering".
Context Engineering: Compaction, Tool Clearing, and Memory
Updated:An agent's window fills with tool output, not with its own thinking. Three API primitives handle that differently: clearing replaces old tool results with stubs, compaction replaces the transcript with a summary, and memory moves notes outside the window. Each one measured on Claude Sonnet 5.5, with what it costs and what it loses.
Long Context, and Where It Breaks: Lost in the Middle
Updated:A model that accepts a long prompt doesn't use all of it equally. The lost-in-the-middle result reproduced on a 1.5B model: a key-value lookup that's perfect at 2,000 tokens and loses things in the middle by 8,000. Why the needle-in-a-haystack test misses it, what RULER and Context Rot measure instead, and the causes: softmax dilution worked out, attention sinks measured, RoPE's reach, and training on short documents. Then what to do about it: put things where they're read, retrieve less, compact, train longer.
Prompt Caching: Paying Once for a Prefix
Updated:API prompt caching is the KV cache and prefix sharing sold as a price. What is actually cached, why only an exact prefix can hit, breakpoints and lifetimes, the write and read prices and the break-even worked from published multipliers, the time a cached prefix saves measured on a 1.5B model, the gigabytes a provider holds for one, how to order a prompt, and how agents break their own cache.
Recursive Language Models: The Prompt as an Environment
Updated:A recursive language model never reads its long input: the input sits in a Python REPL as a variable, and the model writes code to inspect it and calls itself on the pieces. How the idea works, why its three design choices matter, what the paper measured, and a miniature built and run on this blog's whole archive with a 3B model, against direct reading and a fixed map-over-chunks program.