World Models, JEPA and Recursive Self-Improvement: Three Bets Beyond the Next Token

The models in this series mostly learn one thing: predict the next token (What “Generative” Means), then get adapted with fine-tuning and reinforcement learning. This last concept post looks at three directions researchers are betting on for what comes next. They are at different stages, from deployed to speculative, and the post tries to keep each claim at the stage it has actually reached.

Table of contents

World models: learn the simulator

An agent that acts in the world, a robot or a game player, learns from experience, and real experience is slow, expensive and sometimes dangerous. A world model is a learned simulator: given the current state and an action, it predicts what happens next. With one, an agent can practice in its own head.

Ha and Schmidhuber (2018) showed the recipe in miniature: a VAE (VAEs and GANs) compresses each game frame to a small latent vector, a recurrent network predicts the next latent from the current one and the action, and a tiny controller acts on those latents. In VizDoom, a controller trained entirely inside this “dream” transferred back to the real game. The Dreamer line (Hafner et al.; DreamerV3, 2023) made it general: the world model is learned from replayed experience, and the actor and critic of Reinforcement Learning for Language Models are trained mostly on trajectories the model imagines. Like Ha and Schmidhuber’s, Dreamer’s world model predicts in a learned latent space, not in pixels, and is trained partly by reconstructing images; unlike theirs, the latent and the dynamics are learned together rather than one after the other. With one set of settings across domains, DreamerV3 learned to collect diamonds in Minecraft from scratch, without human data (with block-breaking sped up, a change to the game’s settings).

The catch is that a simulator run on its own output compounds its errors. A small network learns one step of a damped pendulum, almost perfectly, and then imagines forward from real starting states. The code in this post runs on the CPU; on an Apple M4 laptop it takes about 3 minutes in all. pendulum_imagination.py trains the one-step model (train, on 20,000 random transitions, three seeds), then steps the real pendulum and the model side by side, the model fed only its own predictions:

for seed in range(3):
    model = train(seed)
    with torch.no_grad():
        torch.manual_seed(100)
        # start anywhere with |angle|, |velocity| <= 2.5: some swings, some spins over the top
        real = imag = (torch.rand(1000, 2) * 2 - 1) * 2.5
        h_i = 0
        for h in range(1, max(horizons) + 1):
            real, imag = step(real), model(imag)
            if h in horizons:
                d = wrap(real[:, 0] - imag[:, 0]).abs()
                errs[seed, h_i], meds[seed, h_i] = (d ** 2).mean().sqrt(), d.median(); h_i += 1

Full script: pendulum_imagination.py

error of imagined angles against the real ones (radians; a random guess would be
off by about 1.8 root-mean-square), 1,000 start states, 3 seeds
        steps imagined        1        5       15       50      100
  RMS error, seed mean   0.0029   0.0148   0.0493   0.1205   0.2679
median error, seed mean   0.0023   0.0114   0.0387   0.0710   0.1448
   RMS, per seed at 15   0.0247   0.0645   0.0586

One step is accurate to 0.003 radians. Fifteen steps out, the imagined angle is off by 0.05 radians (about 3°, and two to three times more in one seed than another); fifty steps out, by 7°; a hundred, by 15°. The error grows roughly in proportion to the horizon: each step’s small error becomes the next step’s starting point, and the imagined pendulum likely drifts out of phase with the real one, like a clock running slightly fast. The median is lower than the root-mean-square, likely because the starts that swing over the top drift fastest.

That is one reason Dreamer imagines only 15 steps ahead from each real state. The other half of the design covers what comes after: the critic estimates the value of the state where imagination stops, and the training targets (λ-returns) blend the rewards imagined along the way with that estimate, so the effective planning horizon, set by the discount, is hundreds of steps. Short imagination plus a learned value is the trade: imagining is cheap, and long imagination is wrong.

How cheap?

# What one Dreamer-style training update imagines. Batch shape and horizon
# follow DreamerV3's defaults: 16 sequences of 64 steps from the replay
# buffer, and 15 imagined steps from every one of those states. Imagination
# runs only the latent dynamics and the actor, critic and reward heads (no
# image encoder or decoder), batched across all start states at once.
B, T, H = 16, 64, 15
starts = B * T
imagined = starts * H
print(f"per update: {starts:,} start states from replay, {imagined:,} imagined steps,")
print(f"computed as {H} sequential batched steps of {starts:,} states each")
# Living through the same number of steps for real, one at a time:
for env_ms, what in ((1, "a fast simulator"), (50, "a robot acting at 20 Hz")):
    print(f"  {imagined:,} real steps in {what} ({env_ms} ms/step): {imagined * env_ms / 1000:,.0f} s")

Full script: dreamer_budget.py

per update: 1,024 start states from replay, 15,360 imagined steps,
computed as 15 sequential batched steps of 1,024 states each
  15,360 real steps in a fast simulator (1 ms/step): 15 s
  15,360 real steps in a robot acting at 20 Hz (50 ms/step): 768 s

One update imagines 15,360 steps, as 15 batched steps of a small latent model, each step computed for all 1,024 states at once. A robot acting at 20 Hz would need nearly 13 minutes to live through the same number of steps. Imagined steps are worse than real ones, so the comparison isn’t one-for-one; the real case for world models is that every real step gets reused many times, which matters most where real experience is slow, costly or unsafe.

There are two ways to use a world model. Dreamer trains a policy inside it, ahead of time. The other way plans at decision time: before each action, try out candidate action sequences in the model and pick the one whose imagined outcome is best (model-predictive control), as robots using V-JEPA 2, below, do.

The same idea now runs at the scale of video. Generative world models such as DeepMind’s Genie 3 (2025) produce explorable scenes at 720p and 24 frames a second from a text prompt, consistent for “a few minutes” by DeepMind’s announcement, and respond to a user’s actions as they go. DeepMind describes Genie 3 as generating frames autoregressively but hasn’t published its architecture; video models of this kind generally build on the generative machinery of Diffusion, for Images and for Text. Whether such models learn physics or a convincing appearance of it is exactly what their consistency over long rollouts tests.

JEPA: predict the meaning, not the pixels

Yann LeCun’s proposal (2022) starts from a criticism of generative models: predicting every pixel of the next frame forces a model to spend capacity on details that don’t matter and can’t be predicted, like the exact pattern of leaves moving in the wind. A joint-embedding predictive architecture (JEPA) predicts in representation space instead. Two encoders embed a context (part of an image, the first seconds of a video) and a target (another part, a later moment); a predictor maps the context’s embedding to the target’s; the loss is the distance between predicted and actual target embeddings. Whatever the encoder chooses not to represent, it doesn’t have to predict.

That freedom has an obvious exploit. If the encoder maps every input to the same vector, the prediction is trivially perfect and the loss is zero. This is representation collapse. Contrastive learning (Embeddings and Contrastive Learning) prevents it with negatives, pushing different inputs apart, at the cost of needing many of them. Non-contrastive methods avoid collapse in one of three ways:

• stop-gradient (SimSiam): treat the target embedding as a constant, so the encoder is only ever moved by the context side;
• an EMA target encoder (BYOL, and I-JEPA): the target side is a slowly updated average of the context encoder’s past weights (an exponential moving average), not trained by gradients at all;
• regularizing the embeddings directly (VICReg): add loss terms that keep each embedding dimension’s variance across a batch above a floor and the dimensions decorrelated, so a collapsed embedding is penalized. LeCun’s own proposal favors this route.

A toy world shows all of it. Each sample has a hidden 8-number state; observations are a fixed random rendering of it into 64 numbers; the two views are independently noised and half-masked. The question is whether an encoder trained only to predict one view’s embedding from the other learns the hidden state. The measure is a linear probe: fit a linear map from embeddings to the hidden state on some samples and score it on others with $R^2$, the share of the hidden state’s variance it explains (1 is perfect, 0 is no better than guessing the average). The probe, VICReg’s two regularizers, and the training loop of jepa_collapse.py’s variants:

def probe(z, s):
    """How much of the hidden state a linear readout recovers from the
    embedding (R^2 on held-out samples): a collapsed embedding scores 0."""
    z = torch.cat([z, torch.ones(len(z), 1)], 1)
    w = torch.linalg.lstsq(z[:2000], s[:2000]).solution
    err = ((z[2000:] @ w - s[2000:]) ** 2).mean()
    return 1 - err / s[2000:].var(0).mean()

def vicreg_terms(z):
    """VICReg's regularizers: keep each embedding dimension's spread across
    the batch above 1, and the dimensions uncorrelated."""
    z = z - z.mean(0)
    std = torch.sqrt(z.var(0) + 1e-4)
    cov = (z.T @ z) / (len(z) - 1)
    off = cov - torch.diag(torch.diag(cov))
    return 25 * torch.relu(1 - std).mean() + (off ** 2).sum() / z.shape[1]

def run(variant, seed, steps=10000):
    torch.manual_seed(seed)
    enc, pred = mlp(D_X, D_Z), mlp(D_Z, D_Z)
    target_enc = copy.deepcopy(enc).requires_grad_(False)
    params = list(enc.parameters()) + (list(pred.parameters()) if variant != "EMA, no predictor" else [])
    opt = torch.optim.Adam(params, 1e-3)
    for _ in range(steps if variant != "untrained encoder" else 0):
        _, ctx, tgt = views(256)
        z_ctx = enc(ctx)
        if variant in ("shared encoder, no tricks", "VICReg regularizers"):
            z_tgt = enc(tgt)                                  # gradients flow into both sides
        elif variant == "stop-gradient":
            z_tgt = enc(tgt).detach()                         # the target side is treated as a constant
        else:                                                 # EMA: a slow copy of the encoder makes targets
            z_tgt = target_enc(tgt)
        p = z_ctx if variant == "EMA, no predictor" else pred(z_ctx)
        loss = ((p - z_tgt) ** 2).sum(1).mean()
        total = loss + (vicreg_terms(z_ctx) + vicreg_terms(z_tgt) if variant == "VICReg regularizers" else 0)
        opt.zero_grad(); total.backward(); opt.step()
        if variant.startswith("EMA"):
            with torch.no_grad():
                for a, b in zip(target_enc.parameters(), enc.parameters()):
                    a.mul_(0.996).add_(0.004 * b)

Full script: jepa_collapse.py

variant                      final loss  embedding std  probe R^2   (mean of 3 seeds, 10,000 steps)
untrained encoder                0.3205       0.055590       0.33
shared encoder, no tricks        0.0000       0.000174       0.07
stop-gradient                    1.3830       0.253997       0.61
EMA target + predictor           1.9868       0.306188       0.61
EMA, no predictor                0.0000       0.000054      -0.00
VICReg regularizers             15.7369       0.927692       0.61

The untrained encoder is the baseline: a random network already passes on some of the hidden state (R² 0.33). Trained without any of the three safeguards, the encoder collapses: the loss goes to zero, the embeddings vary by 0.0002 across inputs, and the probe recovers less than the untrained network did (R² 0.07). With a stop-gradient, an EMA target plus a predictor, or the VICReg terms, the embeddings stay spread out and the probe reaches R² 0.61, nearly double the baseline: the encoder learned what the two views have in common. (No run here measures how much of the hidden state a single masked, noisy view can support, so whether 0.61 is a ceiling is untested.) Removing the predictor from the EMA version collapses it completely (R² 0.00), as BYOL found at scale. Why the first two safeguards work at all, given that the collapsed solution is still a minimum of their loss, is still studied; the leading explanations involve the learning dynamics rather than the loss. VICReg’s terms make collapse expensive in the loss itself.

I-JEPA (Assran et al., 2023) applies this to images: a Vision Transformer (Seeing with Transformers) embeds a large context block, an EMA copy embeds the full image, and a predictor fills in the embeddings of several target blocks, taken from the EMA copy’s output, from the context’s. It learned strong image features with far less compute than pixel-reconstruction methods. V-JEPA (Bardes et al., 2024) does the same across space and time in video, and V-JEPA 2 (2025), a 1.2B-parameter model trained on over a million hours of video, was then turned into a robot’s world model: with its encoder frozen, a new action-conditioned predictor was trained on under 62 hours of unlabeled robot video, and the robot plans by model-predictive control, searching for actions whose predicted embedding approaches that of a goal image. That last step joins the two ideas: a world model that predicts in representation space.

Recursive self-improvement: what exists, and what doesn’t

The oldest of the three ideas is I. J. Good’s (1965): a machine able to design better machines would design a better version of itself, which would do the same, an “intelligence explosion”. It is worth separating what exists from the extrapolation.

What exists: AI systems improving parts of the AI pipeline, with humans in the loop.

• Automated algorithm discovery. AlphaEvolve (Google DeepMind, 2025) pairs Gemini models that propose code changes with automatic evaluators that score them, in an evolutionary loop. Google reported that a scheduling heuristic it found has recovered, on average, 0.7% of its worldwide compute for over a year; that it sped up a key matrix multiplication kernel in Gemini’s architecture by 23%, cutting Gemini’s training time by 1%; and that it found a way to multiply 4×4 complex matrices with 48 scalar multiplications, improving on the best known method for complex-valued 4×4 matrices, Strassen’s from 1969. On over 50 open mathematical problems it matched the best known solutions in about three-quarters and improved them in a fifth. Every result depended on a problem with a fast, automatic way to score a candidate.
• Self-modifying agents. Sakana’s Darwin Gödel Machine (2025) is a coding agent that edits its own code and keeps the variants that score better on a coding benchmark; it reported raising its own SWE-bench score from about 20% to about 50%. The model underneath stayed fixed; what improved was the agent’s tooling and workflow. The same paper reports the system gaming its evaluation, faking logs of tool use it hadn’t done: a self-improver optimizes the evaluator it’s given.
• Models generating their own training data, which is most of GRPO and Reinforcement Learning from Verifiable Rewards: a model samples solutions, a verifier checks them, and training reinforces what passed.

One of these is a real, if small, closed loop: AlphaEvolve made the training of Gemini, the model family that powers it, about 1% faster.

What is speculation: an unbounded loop. None of these systems improves the thing that improves it without limit. Each optimizes against a fixed evaluator that humans chose, gains are bounded by what that evaluator can measure, and the loop stops where automatic evaluation stops: a kernel’s speed is easy to score, a research direction is not. Whether those limits fall away as models improve, and how fast, is the open question behind much of the debate about advanced AI. It isn’t something present systems demonstrate, in either direction.

Where this leads

• The Notebook Assistant, End to End closes the series on the ground the three bets start from: one working system built from next-token models, retrieval and a fixed workflow, with its measured costs and failures.

Takeaways

• A world model is a learned simulator an agent can practice in, or plan with. Imagined rollouts are cheap and accumulate error: a learned pendulum drifted about 3° in 15 steps and 15° in 100, one reason Dreamer imagines 15 steps and lets a value function cover the rest.
• JEPA predicts embeddings, not pixels, so the encoder can ignore what’s unpredictable. The cost is a trivial solution, collapse, which the naive version found on a toy world; a stop-gradient, an EMA target with a predictor, or VICReg’s variance terms avoided it (linear-probe R² 0.61, against 0.33 untrained and 0.07 collapsed).
• I-JEPA and V-JEPA apply this to images and video; V-JEPA 2, with an action-conditioned predictor, plans robot actions in embedding space.
• Self-improvement exists today as AI optimizing parts of AI under automatic evaluators (AlphaEvolve’s 0.7% of Google’s compute, 1% of Gemini’s training time). An unbounded loop is speculation that current systems neither show nor rule out.

References

Start with Ha and Schmidhuber’s World Models and DreamerV3, then LeCun’s position paper and I-JEPA; AlphaEvolve’s announcement is the clearest account of what automated self-improvement has done so far.

• Ha and Schmidhuber, World Models (2018): the VAE-plus-recurrent world model and the controller trained inside its “dream”.
• Hafner et al., Mastering Diverse Domains through World Models (DreamerV3, 2023; Nature, 2025): actor and critic trained on 15-step imagined rollouts with λ-returns, and the Minecraft diamonds.
• LeCun, A Path Towards Autonomous Machine Intelligence (2022): the case for JEPA against pixel prediction, and for regularized embeddings.
• Grill et al., Bootstrap Your Own Latent (BYOL, 2020); Chen and He, Exploring Simple Siamese Representation Learning (SimSiam, 2020); Bardes, Ponce and LeCun, VICReg (2021): the three safeguards against collapse that the toy compares.
• Assran et al., Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (I-JEPA, 2023): JEPA on images, with an EMA target and block masking.
• Bardes et al., Revisiting Feature Prediction for Learning Visual Representations from Video (V-JEPA, 2024): the same across space and time in video.
• Assran et al., V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025): the 1.2B video model turned into a robot’s world model for model-predictive control.
• Google DeepMind, Genie 3: A new frontier for world models (August 2025): the 720p, 24-frames-a-second explorable scenes.
• Google DeepMind, AlphaEvolve (2025): the compute, kernel and matrix-multiplication results of automated algorithm discovery.
• Zhang et al., Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents (2025): a coding agent that edits its own code, and its gaming of the evaluator.
• Good, Speculations Concerning the First Ultraintelligent Machine (Advances in Computers, vol. 6, 1965): the original argument for an “intelligence explosion”.