Tag: decoding
All the articles with the tag "decoding".
Decoding: From a Distribution to Text
Updated:A language model outputs a probability for every possible next token; decoding is the policy that turns that into text. Greedy and beam search, temperature derived, top-k, top-p and min-p on a real model's distributions, the order the steps run in, repetition penalties, and why the same prompt can give different text. Each one measured with Qwen2.5-1.5B on twenty paragraphs of this blog, against what a person actually wrote next.