Posts
All the articles I've posted.
Dropout and Normalization: Two Layers Every Diagram Has
Updated:Two layers that sit in almost every network diagram, and what each one actually does: dropout as a regularizer, derived and tested (on its own it makes the router overconfident; alongside weight decay, and in long training, it helps), then normalization, from BatchNorm and why language models can't use it, to LayerNorm and RMSNorm with worked vectors and hand-written backward passes, what normalizing does to the gradient and the learning rate, and what a norm layer costs.
Evaluating and Observing Agents: Outcomes, Trajectories, Traces and Spans
Updated:An agent's answer can be right for the wrong reasons, wrong for a trivial one, or right only some of the time. Outcome, trajectory and tool-use evaluation; SWE-bench and τ-bench; pass@k against pass^k, with the arithmetic and unbiased estimators; and a small agent instrumented with OpenTelemetry's GenAI span conventions and evaluated from its traces, where most 'failures' turned out to be the answer's format or the harness's own token budget, and the traces showed exactly which.
Fine-Tuning, and What It Destroys
Updated:Fine-tuning moves a pretrained model a short distance in weight space, and how far it moves decides what it gains and what it forgets. Continued pretraining vs supervised fine-tuning, the SFT loss with prompt masking worked on a real chat template, learning rates and memory. Then Qwen2.5-0.5B fine-tuned on this blog, two seeds per setting, with what it learns and what it forgets measured side by side; replay as the remedy, EWC derived, and when fine-tuning is the wrong tool.
Many Agents: Subagents, Communication and Orchestration
Updated:Splitting a task across agents buys clean contexts and parallelism, and costs coordination and lost information. Why subagents exist; roles; communication topologies; the A2A protocol; the standard orchestration patterns; a token-cost model, on real document sizes, of one agent against a lead with six workers (a context a quarter to two-fifths the size and far fewer sequential steps whatever the assumptions, a token bill that can go either way, and only 8% of the source reaching the lead); and the published evidence on when multi-agent systems win and when they lose.