After LLMs: the bet on world models

Earlier this year Yann LeCun, Turing Award winner and for a decade Meta's chief AI scientist, left the company and founded a startup of his own, AMI Labs, headquartered in Paris. In an interview that made waves he explained why: language models, the technology the entire industry is betting on, will not lead to human-level intelligence. A different architecture is needed. In the interview he also points to which one, and recommends a specific paper to read. We spent a few weeks reading that paper and its theoretical foundation, line by line: this note is the map of what we found, together with the objections that remain standing.

The thesis: language is not the world

LeCun's argument should be taken seriously because it is not an offhand opinion: it is built step by step. Language models work when the problem is made of words: text, code, formal mathematics — there language does not describe the problem, it is the problem. But the physical world is continuous, high-dimensional, noisy; a cat, LeCun observes, plans actions in the world without possessing any language. Saying that LLMs are a dead end does not mean saying they are useless: it means they are not the road to general intelligence.

In their place, three properties an intelligent architecture should have. First, predict the consequences of one's own actions: this is the operative definition of world model. Second, plan by search and optimization — mentally trying out sequences of actions and picking the best one — instead of generating one word at a time hoping the sentence lands somewhere. Third, predict at an abstract level of representation, not at the pixel level: foreseeing that the ball will bounce, not the RGB value of every pixel of the next frame. On this third point LeCun is blunt: generative approaches applied to the real world, from VAEs to models that reconstruct images, have been "deeply disappointing". And a 2024 paper from his group tries to formally prove why: learning by reconstructing pixels produces poor features for perception.

JEPA: predicting representations, not pixels

The alternative architecture is called JEPA, Joint Embedding Predictive Architecture, and the idea fits in one sentence. Take two views of the same input — an image and a corrupted version of it, or two instants of a video — and train a network, the encoder, so that from the numerical representation of one view it can predict the representation of the other. Not the pixels, not the words: the internal representation, a list of numbers whose meaning lies in the collective geometry.

If it succeeds, the encoder has learned something abstract and robust about the world, without anyone attaching a label to it. This is self-supervised learning: the signal of correctness comes from the data itself, not from human annotators. It is the same reason LLMs were able to swallow the entire web; here, though, the task is not "guess the next word", it is "guess how your representation of the world changes".

All these representations live in a latent space: an abstract space with hundreds of dimensions where every input is a point and the geometry carries meaning — cats form one region, trucks a distant region. It is inside this space that a world model "reasons", making predictions and planning. Which is why its organization matters enormously. And this is where the story takes an interesting turn.

To keep something concrete in front of us, let us set aside an example we will use all the way through: PushT, a table seen from above that resembles a game of air hockey. A small "hand" moves across the table and must push a T-shaped block until it covers a target silhouette. It is one of the testbeds on which the model we are discussing is trained: every frame of the table becomes, for the model, a point in 192 dimensions in its latent space.

The enemy: collapse

The prediction task has a logical bug. Training a neural network does not pursue good representations: it pursues only a low error measure. And the prediction error has a disastrous shortcut: if the encoder always produces the same output, whatever it sees, predicting that output is trivial, the error goes to zero, training has "won" — and the network has thrown away all the information. This is called collapse, and it is not a rare accident: the gradient actively pushes toward it, because the stupid solution is easier to reach than the intelligent one. There is also an insidious variant, dimensional collapse, in which the points do not end up in a single point but flatten onto a plane, wasting nearly all the available dimensions.

For ten years the problem has been kept at bay with an accumulation of tricks: stop-gradient, teacher-student pairs updated with a moving average, forced normalizations, contrastive negative examples. They work, but they are heuristics: nobody can really explain why they work, they require careful tuning, they are fragile, and every project combines them differently. LeCun himself, speaking of the family of methods that includes DINO, sums it up like this: it works, but we do not know why.

This is where the first paper arrives, LeJEPA by Randall Balestriero and LeCun (November 2025). Instead of piling on another trick, the authors derive the design from a principle, with a result that is mathematically proven, not merely observed in experiments: without knowing which task they will serve tomorrow, the optimal distribution of representations is the isotropic Gaussian — the cloud of points in latent space must be spherical, identical in every direction, like a round pile of sand rather than a flattened streak. If the cloud is deformed, for any future task it costs more errors and more uncertainty than the spherical one. The spherical shape wastes no direction.

What remains is to enforce it, in practice, during training. The tool is called SIGReg and works like this: random directions are drawn in the space, the representations are projected onto each of them, reducing them to simple lists of numbers, and one measures how much each list resembles a Gaussian, with a statistical test chosen because it behaves well during training (Epps-Pulley, which compares the Fourier transforms of the two distributions). A classic theorem, Cramér-Wold, guarantees the trick works: if all one-dimensional projections are Gaussian, the whole cloud is too. The entire mechanism fits in about twenty lines of PyTorch, without a single parameter to tune inside it.

The experimental results cover the promises one by one. Tested on about fifty architectures from eight different families, no configuration leads to collapse; removing the teacher-student and every other crutch, the representation holds anyway, proving that SIGReg does the dirty work. It needs only 128 examples at a time, where contrastive methods demand thousands. Trained from scratch on a specialist domain — galaxy morphology, eleven thousand images visually far removed from natural photos — it clearly beats DINOv2 and DINOv3, the "frontier" models adapted from the outside: training on your own domain stops being impractical. And the most surprising result is almost a technical detail: the training loss correlates at 99% with the real quality of the representations. Those who work in the field know what this means: until now, to know whether a self-supervised model was learning something useful you had to stop and measure it with labeled data — precisely the ones you wanted to avoid. With LeJEPA the curve you watch during training tells the truth.

How a world model works, piece by piece

LeJEPA learns to see: static representations, perception. The second paper, LeWorldModel (Maes, Le Lidec, Scieur, LeCun, Balestriero — March 2026, the one recommended by LeCun in the interview), makes the leap we care about: it learns how the world changes. It is worth stopping and taking the mechanism apart, because it is simpler than it looks. Three pieces, and the PushT table as the example.

First piece: the encoder, the mental chessboard. Every frame of the table enters the encoder and comes out as a list of 192 numbers. The individual numbers mean nothing readable; together they are the "position on the chessboard": where the hand and the block are, how the block is oriented. A 150,000-pixel image compressed into 192 numbers that contain what matters — and only that.

Second piece: the predictor, the rules of the game. The second module receives the latest positions on the chessboard plus a candidate move, and answers with the next position: "if you push like this, the block ends up there". It draws nothing, produces no images: it speaks only the language of the 192 numbers. This is where the leap from encoder to world model happens — action. JEPA's "two views" are no longer an image and its corrupted version: they are two consecutive instants, and in between there is something the player did. The model does not learn "what comes next": it learns "what happens if I take this action".

How it learns: by watching recorded games. Trajectories of frames and moves collected with no quality criterion whatsoever, without rewards and without ever generating a pixel: the training recipe is LeJEPA's, unchanged — prediction error in latent space plus SIGReg. It does not learn to win: it learns the rules. Winning comes later. And the whole model fits in 15 million parameters — three thousand times fewer than a mid-sized LLM — and trains on a single GPU in a few hours.

Third piece: planning, the chess master. At this point playing a real game works the way a chess player thinks. You give it the snapshot of the table now and the snapshot of the goal; the encoder translates them into two positions on the mental chessboard. Then the model imagines: it proposes thousands of possible move sequences, dreams them all in parallel inside the predictor — without touching the real table — and for each one measures how close the imagined arrival point is to the goal. It keeps the best sequences, resamples around them, repeats until convergence (this is the Cross-Entropy Method). And finally — a detail that matters — it does not execute the whole plan: it executes only the first move, looks at the real table again, and starts dreaming anew. Because the dream drifts: the further ahead you imagine, the more the simulation detaches from reality; better a short dream redone often than a long one believed to the end.

The contrast with LLMs is total. A language model would write the game one move at a time, each one irrevocable, without ever comparing alternatives. Here entire sequences are searched inside a simulation and measured. The authors report that planning this way is up to 48 times faster than with world models built on top of frozen foundation models — credit to a latent space hundreds of times more compact.

The objections, and what remains open

So far the charitable reconstruction. Now the objections, gathered from industry discussions and from the limits the papers themselves admit.

The world is chaotic. In a chaotic system the error grows exponentially with the horizon: predicting is impossible beyond short thresholds. The JEPA line's answer is that one does not predict the trajectories of pixels but their abstract consequences — not where every leaf will be, but that the tree will fall. Partially convincing; and honestly the LeWorldModel paper itself admits that autoregressive simulations accumulate error as the horizon stretches. We measured it ourselves: the result, in the second part, is more interesting than we expected.

Predicting is not understanding. A model can anticipate statistically without having any causal model. This is the deepest objection, and the tools to test it exist: one probes the latent space about hidden physical quantities, one measures the model's "surprise" in front of physically impossible events. These are experiments that can be done at home — we did them, and they are the heart of the second part.

LLMs already have a world model. Implicit, perhaps: inside the weights of a language model there is a great deal of knowledge about the world. But planning by optimization requires a model that can be queried with candidate actions and that returns comparable outcomes; knowledge buried in the weights, accessible only by making the model talk, does not lend itself to this use.

The cost. The objection has overturned itself: fifteen million parameters on one GPU against the billions of generative video models. If the JEPA line has a cost problem, this is not where it is. If anything the cost is political: while the main laboratories pour billions into language models, this research advances with the means of a university group.

Nothing new. Partly true: the idea of maximizing information in representations dates back to 1989, and JEPAs descend from a long genealogy. What is new is the theoretical foundation: for the first time the anti-collapse mechanism is not a trick that works, but the consequence of a proof.

What to watch from here on

Three signals, over the next twelve to eighteen months. The first is AMI Labs: LeCun declares that by early 2027 the paradigm shift will be "obvious to everyone". It is the prediction of someone betting his career on it: it should be taken with that caution; but it is verifiable, and it has a date. The second is the outcome of physical probing experiments on LeWorldModel: if the latent space really encodes physical quantities never shown explicitly, the "predicting is not understanding" objection weakens considerably. The third is the rest of the world on the move: Fei-Fei Li with World Labs on spatial "large world models", NVIDIA with world models for climate, rumors of an OpenAI repositioning toward robotics. Whether the right approach is JEPA or another, the industry's direction of travel — from models that talk to models that predict the world — is now visible to everyone.

One last methodological note, which we care about. Both papers are public with code, weights and complete recipes: anyone can rerun everything. After spending two days verifying a robotics announcement with no paper, no weights and no independent witnesses, the difference in epistemic status is the first thing we feel we should point out. On this road every step can be checked; and we will keep doing so.

Notes and references

The starting point is the interview Yann LeCun on What Comes After LLMs, podcast Unsupervised Learning (2026): the thesis on the limits of LLMs, the three properties of intelligence, AMI Labs and the explicit recommendation of LeWorldModel.

The papers discussed:

• R. Balestriero, Y. LeCun, LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics, 2025 — arXiv:2511.08544. The theoretical foundation: why the isotropic Gaussian and how SIGReg enforces it. Code: github.com/rbalestr-lab/lejepa.
• L. Maes, Q. Le Lidec, D. Scieur, Y. LeCun, R. Balestriero, Stable End-to-End Joint-Embedding Predictive Architecture from Pixels (LeWorldModel), 2026 — arXiv:2603.19312. The world model: end-to-end training from pixels and planning in latent space. Code: github.com/lucas-maes/le-wm.
• R. Balestriero, Y. LeCun, Learning by Reconstruction Produces Uninformative Features for Perception, 2024 — arXiv:2402.11337. The formal argument against generative approaches cited in the first section.
• R. Balestriero, N. Ballas, M. Rabbat, Y. LeCun, Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density, 2025 — arXiv:2510.05949. The link between Gaussian embeddings and density estimation.

The objections of the fifth section come from public industry discussions (in particular r/artificial, 2026) and from the parts of the papers that flag their own limits. This is the first part of a series of two: in the second, the same claims are measured in-house, experiment by experiment, on the public model. (Updated 2026-08-31: completed a full read of LeWorldModel and rewrote the section on how the model works, which in the first version remained too compressed.)