Tag: reinforcement-learning
All the articles with the tag "reinforcement-learning".
GRPO and Reinforcement Learning from Verifiable Rewards
Updated:When an answer can be checked, the reward model goes away, and sampling a group of answers per question and comparing them within the group gives the advantage without a value network. Verifiable rewards, GRPO's group-relative advantage worked on four samples, its objective and KL estimator, the memory it saves, why groups that all agree teach nothing, and the length bias Dr. GRPO identified, measured on a toy where length never affects the reward.
Learned Orchestration: A Case Study of Sakana Fugu
Updated:What if the orchestrator of a multi-agent system were itself a model, trained to pick and instruct frontier models? Sakana's Fugu report (2026) answers with two designs: a router that reads one hidden state and picks one worker, trained by supervised fine-tuning and then evolution strategies, and a Conductor trained with GRPO to write whole multi-agent workflows in natural language. How each is built, the failure called orchestration collapse and its fix, and the reported results read critically: ahead of the best single model on 7 of 11 benchmarks each, against baselines gathered from providers and leaderboards rather than run under the same conditions.
Reinforcement Learning for Language Models: From Policy Gradient to PPO
Updated:When a score isn't differentiable, a model can still climb it with the log-derivative trick. Reinforcement learning's vocabulary mapped onto text, REINFORCE derived in three lines, why a baseline is free and essential, advantages and GAE, and PPO's clipped objective worked case by case. Then REINFORCE, REINFORCE with a baseline and PPO trained on a tiny autoregressive policy, with controls, ten seeds at two learning rates, and the memory bill for PPO on a language model: four models at once.
World Models, JEPA and Recursive Self-Improvement: Three Bets Beyond the Next Token
Updated:Three research directions for what comes after next-token prediction, each with what is shown and what is hoped. World models learn a simulator and train or plan inside it: a learned pendulum imagined forward drifts 3° in 15 steps and 15° in 100, one reason imagination horizons are short. JEPA predicts embeddings instead of pixels: on a toy world, the naive version collapses to a point, and a stop-gradient, an EMA target with a predictor, or VICReg's variance terms prevent it. And self-improvement: what systems like AlphaEvolve have actually done, separated from what remains speculation.