Tag: reasoning
All the articles with the tag "reasoning".
GRPO and Reinforcement Learning from Verifiable Rewards
Updated:When an answer can be checked, the reward model goes away, and sampling a group of answers per question and comparing them within the group gives the advantage without a value network. Verifiable rewards, GRPO's group-relative advantage worked on four samples, its objective and KL estimator, the memory it saves, why groups that all agree teach nothing, and the length bias Dr. GRPO identified, measured on a toy where length never affects the reward.
Reasoning Models and Test-Time Compute
Updated:A reasoning model spends generated tokens as compute: trained with verifiable rewards, it learns to think before answering, and gets better the longer it thinks. What test-time compute means, how RL produces long chains and distillation passes them to small models, thinking budgets, and what 1,000, 10,000 and 50,000 thinking tokens cost in KV cache and time on two real models. Then a 1.5B model on checkable word problems, two seeds: answering directly, reasoning step by step, and voting over up to 16 chains, with the tokens each one costs, then on harder eight-step problems, where voting stops helping and a verifier could still gain.