About

Residual Stream is a personal notebook on LLM internals and AI engineering. It exists mostly so that things I’ve had to re-learn more than once — how attention actually works, why KV caches matter, what paged attention is solving, how quantization trades off precision for memory — end up written down somewhere I can find them again.

Expect posts on:

Transformers — LLM fundamentals and internals, tokenization, positional encodings, Q/K/V matrices, transformer architecture
Attention — self-attention, multi-head attention, causal masking, cross attention, grouped query attention
Inference & serving — KV cache, paged attention, speculative decoding, continuous batching, prompt caching, mixture of experts, vLLM
Context engineering — prompt engineering, prompt chaining, chain of thought, RAG, vector databases
Efficiency — fine-tuning, PEFT, LoRA/QLoRA, quantization, model compression, distillation, small language models
Agents — ReAct, agentic AI, tool use, memory, MCP, agent architecture, subagents, multi-agent systems, orchestration
Evaluation — evaluating LLMs and agents, LLM-as-a-judge
Multimodal AI

Posts often include math (rendered with KaTeX), diagrams, and small interactive visualizations where a static image would leave out the interesting part.

About the author

I’m Tan Le, an AI engineer. I write these posts to learn the field in depth myself, tracing each idea back to its paper and running the code where I can, and I hope they’re useful to anyone starting the same path.

This site is built with Astro on top of the AstroPaper theme.