Tag: policy-gradient

All the articles with the tag "policy-gradient".

• Reinforcement Learning for Language Models: From Policy Gradient to PPO

  Updated:19 Oct, 2026

  When a score isn't differentiable, a model can still climb it with the log-derivative trick. Reinforcement learning's vocabulary mapped onto text, REINFORCE derived in three lines, why a baseline is free and essential, advantages and GAE, and PPO's clipped objective worked case by case. Then REINFORCE, REINFORCE with a baseline and PPO trained on a tiny autoregressive policy, with controls, ten seeds at two learning rates, and the memory bill for PPO on a language model: four models at once.