Posts
All the articles I've posted.
Giving Agents Tools and Knowledge: MCP, Skills and OKF
Updated:Three answers to one question, how an agent gets capabilities without bloating its context: the Model Context Protocol for live tools and data, Skills for procedures loaded on demand, and the Open Knowledge Format for curated knowledge. A working MCP server for this blog, called through the SDK and as raw JSON-RPC under the stateless 2026-07-28 protocol; progressive disclosure's token arithmetic; an OKF bundle generated from the posts; and the security model all three share.
Mixture of Experts: More Parameters, Same FLOPs
Updated:A mixture of experts replaces a Transformer's MLP with many MLPs and a router that sends each token to a few of them, so a model can hold far more parameters than it uses per token. The router worked through, the load-balancing loss derived, DeepSeek-V3's bias balancing, capacity and dropped tokens, and shared and fine-grained experts. Then an MoE version of the tiny GPT from Build a Tiny GPT, End to End, and Train It on This Blog, trained four ways, the parameter counts of Mixtral and DeepSeek-V3 recomputed from their configs, and what experts cost at serving time: memory, all-to-all traffic, and batches big enough to keep experts busy.
Running LLMs on a Laptop: GGUF and llama.cpp
Updated:Local inference is the quantization and memory-bandwidth story at consumer scale: GGUF packages a quantized model in one memory-mappable file, and llama.cpp turns bytes per second into tokens per second. Inside a real GGUF file, what K-quant names like Q4_K_M mean, and tokens per second measured on this laptop's GPU and CPU against the bandwidth prediction, with what each quantization level costs in quality.
TensorRT-LLM: Compiling Inference Engines Ahead of Time
Updated:TensorRT-LLM began by compiling each model into an engine: operations fused into fewer kernels, and every kernel timed and chosen for the GPU and the shapes it would serve. Both steps measured on a laptop GPU: fusion removed 26% of the operations per token and 7% of prefill time; the best version of one kernel depended on the shape, and the worst cost twice as much. Then why NVIDIA moved the project onto a PyTorch runtime, and what it keeps of the compiled design.