Tag: self-supervised-learning
All the articles with the tag "self-supervised-learning".
World Models, JEPA and Recursive Self-Improvement: Three Bets Beyond the Next Token
Updated:Three research directions for what comes after next-token prediction, each with what is shown and what is hoped. World models learn a simulator and train or plan inside it: a learned pendulum imagined forward drifts 3° in 15 steps and 15° in 100, one reason imagination horizons are short. JEPA predicts embeddings instead of pixels: on a toy world, the naive version collapses to a point, and a stop-gradient, an EMA target with a predictor, or VICReg's variance terms prevent it. And self-improvement: what systems like AlphaEvolve have actually done, separated from what remains speculation.