Posts
All the articles I've posted.
The Transformer's Three Attention Patterns: Self, Masked, and Cross
Updated:Self-attention, masked attention and cross-attention are one computation, softmax(QKᵀ/√d)V, with Q, K and V drawn from different places. Derived from a weighted average, worked through on small matrices, then checked on real models: Qwen2.5-1.5B's mask hides the future exactly, its heads learn to look one token back or at the first token, a translation model's cross-attention lines German words up with English ones, and the cost grows as the square of the length.
VAEs and GANs: Two Ways to Sample What You Can't Write Down
Updated:A language model samples by writing the distribution down one token at a time. For images that means hundreds of thousands of sequential steps in an arbitrary pixel order, so two other answers grew up: learn an encoder and maximize a bound on the likelihood (the VAE), or learn a critic and fool it (the GAN). The ELBO derived and checked exactly on a model small enough to solve, the reparameterization trick, the KL term worked by hand, the GAN's optimal discriminator and its Jensen-Shannon value checked numerically, and all three models (VAE, GAN, WGAN-GP) trained on the same eight-mode test, where off-mode samples, a dropped mode and an outright training failure all show up, with a test of where the VAE's off-mode samples come from.
Vector Search: Exact Search and the Approximate Indexes
Updated:Finding the nearest embeddings is easy and linear; every index trades a little recall for a lot of speed. Similarity metrics and when they agree, why trees fail in high dimensions, and LSH, IVF, product quantization with a worked 8-byte code, and HNSW as a navigable small world, with an interactive graph search. Then all of them measured with FAISS on 71,865 real sentence embeddings: recall against latency and bytes per vector.
vLLM Architecture: Continuous Batching and Serving at Scale
Updated:A serving engine is a loop that decides, every forward pass, which requests' tokens go into it. What a pass costs on a real GPU, measured; static batching against continuous batching; vLLM V1's scheduler, where prefill and decode are one rule under a token budget, written out and run on ShareGPT traffic with the measured costs, with paged memory and preemption; the processes vLLM runs; tensor parallelism's communication, worked for a 70B model; and serving many LoRA fine-tunes from one model, measured.