Posts
All the articles I've posted.
Guardrails and Prompt Injection: Why Filters Reduce Risk and Only Architecture Bounds It
Updated:A language model reads instructions and data through the same channel, so text in a retrieved page can act as a command. Input and output guardrails and classifier filters; direct versus indirect injection; why there is no escaping, as there is for SQL; and an injection attack measured against a small tool-using assistant that summarizes this blog, before and after four defenses, from a system-prompt rule and spotlighting to an input classifier and removing the tool.
How Coding Agents Work: Claude Code and Cursor, from Their Documentation
Updated:Coding agents are the most demanding agent harnesses in production: large codebases, many tools, real side effects, and a definition of done that can actually be executed. How Claude Code and Cursor work, using only what their documentation and engineering posts state: the gather-act-verify loop, searching a codebase with grep and with an embedding index (and why Cursor dropped the index), verification, project memory, context management, permissions and checkpoints, subagents and hooks (with a working hook), and Cursor's tab and apply models.
KV Cache Compression: Eviction and Quantization
Updated:After sharing key-value heads, two levers are left for the KV cache: store fewer tokens, or store each in fewer bits. Both are lossy in different ways. The budget worked for Llama-2-13B, eviction from sliding windows to attention sinks, heavy hitters and SnapKV, and per-token versus per-channel quantization as in KIVI; then both measured on a real model's loss on held-out text, against the full cache.
The KV Cache: How Transformers Avoid Recomputing the Past
Updated:Generating text recomputes every earlier token's keys and values unless they are kept. The KV cache keeps them, and it is exact: in Qwen2.5-1.5B-Instruct the next token matched at 32 of 32 steps. It turned a step at 4,096 tokens from 4.7 s into 65 ms on a laptop. Then what it costs: 28 KiB per token for Qwen, 800 KiB for Llama 2 13B, why decoding is limited by memory reads, how batch size trades latency for throughput, and how GQA, MLA and cross-layer sharing shrink the cache.