Why the most impressive generated worlds arrive without a report card
Google DeepMind released Genie 4 in October, a world model that generates interactive environments in real time. A few weeks before that, AMD agreed to acquire World Labs, Fei-Fei Li's spatial intelligence company, in an all-stock deal valued at around $8.2 billion. Whatever else world models are at this point, they're being valued as infrastructure rather than as demos.
World Labs announced Atlas in September, and it's a genuinely good piece of work from people who've earned the benefit of the doubt: Fei-Fei Li, Justin Johnson, and Ben Mildenhall, who created NeRF. Atlas takes camera geometry as a real input rather than a text description of where you'd like the camera to go, it outputs actual 3D in the form of point clouds and Gaussian splats, and it can rebuild a space from two or three photographs.
The idea underneath it is elegant. Language models are trained to predict the next token, video models to predict the next frame, and Atlas is trained to predict the next view: what you would see if the camera moved over there. To get good at that, a model has to pick up geometry, occlusion and structure along the way, whether you asked it to or not. That framing is right, and this series has been making a version of the same argument since 2025, in Building the Foundation for Spatial Intelligence and again in The Geometry of Common Sense.
Go looking for the evaluation, though, and there isn't much of one. Atlas shipped with no paper, no model card, no code and no published latency. Every benchmark in the announcement was run by World Labs itself, and in the camera-control comparison Atlas was given real camera trajectories while the competing models were given text descriptions of the same movements, a mismatch the company is upfront about. On reconstruction the results are mixed rather than a sweep: ahead of one baseline, tied on a second benchmark, behind another model on a third.
None of which is a failing on World Labs' part. There wasn't an independent evaluation available to run. We've reached the point where a well-funded lab can ship a model that builds whole worlds, and nobody, the lab included, has an agreed way of saying how true those worlds are.
So who grades the world model?
What a flight simulator gets right
Pilots have trained in simulators for decades, rehearsing engine failures and frozen runways that would be lethal or ruinously expensive to stage for real. But nobody builds a flight simulator by asking an image generator what a plane probably does in a storm. It works because it's built on measurements taken from real aircraft, and everybody involved knows which parts are measured and which are approximated.
Generated worlds need that same backbone, and when they don't have it the failure is a quiet one. A generated kitchen can look completely convincing, with the light falling correctly and the glass reflecting the room, while the cup slowly changes size as the camera moves past it and the handle vanishes whenever something crosses in front of it. In a movie nobody notices. In a robot deciding where to close its fingers, it's the whole ballgame. A film set only has to look right from where the camera stands, and a robot needs the back wall too.
The scores we normally reach for won't catch this either. An image-quality metric tells you a view looks like a photograph. It doesn't tell you the table is flat.
The photocopy problem
This matters more every year, because generated data increasingly goes on to train the next system rather than just entertaining us.
Mostly that's a good thing. Synthetic data creates rare situations, varies the weather, balances out a lopsided dataset and lets an agent rehearse something dangerous safely. Some reconstruction models train beautifully on procedurally generated geometry and still transfer to real objects. Anyone who tells you synthetic data can't teach reality is overstating it, earlier posts in this series included.
The narrow danger is generated data being asked to validate itself. Think about making a photocopy of a photocopy: by the tenth generation the faint edges are gone and the small defects have hardened into features. Nothing new was created along the way, the machine just kept amplifying its own reading of the original. If a model learns a slightly wrong relationship between a cup and its reflection, and its output trains the next model, that small error quietly becomes a belief about how the world works. A student can learn a great deal from exercises written by a teacher, but if the student writes the questions, answers them and marks the exam, confidence grows a lot faster than knowledge.
Ask it for a picture you already have
Here's the genuinely surprising part: grading a world model isn't an unsolved research problem. It's unbuilt plumbing. The technique is almost embarrassingly simple.
Ask the model to predict something you already know, then check whether it got it right.
Give it one view of a scene and ask for a second. Give it two and ask for a viewpoint you deliberately held back. The answer already exists, either recorded by a camera or rendered by an engine, so you can set the model's imagination next to the real thing and look. Did the cup keep its shape and its size? Did the reflection move the way it should have? Did the hand that went behind the chair come back out in the right place? A model can produce a completely convincing picture and still fail all three, and that's exactly what makes the test useful. It means the model learned how things look without learning how they're arranged.
The answers you check against come from two places.
A renderer gives you perfect ground truth, because somebody authored the scene: exact depth, exact occlusion boundaries, and a camera you can put anywhere you like. That last part is the valuable one. You can place the held-out camera far away from the input views and force the model to show you surfaces it never saw, which is where hallucination likes to hide and what nobody catches by looking at the screen.
But a renderer can only give you scenes somebody sat down and authored. It won't hand you wet asphalt, or a reflection in a shop window, or a phone sensor that dropped a frame halfway through.
A stereo pair will, and the reason is fairly mundane: the second image is the answer key. Show the model the left eye, ask it for the right, compare. Any single result is weak on its own, but across a few million captures a systematic drift in the geometry shows up as a shift in the distribution. Two views of the same instant also sidestep something video can't. If the scene itself is moving, a video hands you two views of two different moments and leaves you to untangle which thing changed, the camera or the world. A stereo pair never had that ambiguity in the first place.
Now, the obvious objection. If those captures already exist, they were probably in the training set, and testing a model on its own training data measures memory rather than understanding. The fix is old and unglamorous: hold some back and never train on them, splitting by scene, place and time rather than at random, because two clips of the same kitchen five minutes apart aren't independent samples. A renderer needs no holdout at all, since it's a generator rather than a dataset, and the thousand kitchens it makes tonight were in nobody's training data because they didn't exist until we asked.
There's one more consequence, and it's easy to skate past. A model can't generate its own answer key, because the moment it does, it isn't an answer anymore. Training data can be synthesized; the thing you check against can't be. So the scarce ingredient was never the raw number of captured views, and Atlas rebuilding a room from three photographs makes that clear enough. What stays scarce is real, simultaneous, untrained-on observation of the messy world.
Geometry isn't physics
All of this checks whether a model's geometry is self-consistent. None of it says anything about mass, friction, or whether a surface will hold weight. A model can be perfectly consistent from every angle and still have no idea the glass is about to shatter. That's a harder problem, and a later one.
Atlas introduces a second wrinkle, and this one is easy to miss. For fifty years, reconstructing the world and generating new imagery were separate problems solved with separate tools, and Atlas folds them into a single model: it reconstructs what the observations support, and generates plausible content where they run out. That's a real achievement, and it quietly removes a seam we used to get for free. When one network both measures and invents, which parts of the output were observed and which were imagined? The old pipeline made that boundary legible almost by accident. A unified model won't, unless somebody deliberately builds it to. Knowing whether a given surface was measured, inferred or dreamed is going to matter a lot more than it does today, and it will probably end up being something a model reports rather than something anyone has to guess at.
Models can dream, reality keeps score
World models are becoming extraordinary engines of experience. They'll put robots inside warehouses nobody built, stage accidents nobody would dare reproduce, and let an agent live a thousand lifetimes before its first real action. Atlas is a serious step in that direction, and new-view prediction may well prove as load-bearing as its authors think.
But a model can imagine what might happen, and it can't certify what did. That has to keep coming from contact with the world: synchronized viewpoints, calibrated sensors, physical interaction, and a reserve of observation that nobody trained on.
None of this needs new science. It needs somebody to build the held-out sets, publish the protocol and run the comparison on equal terms. And the stakes aren't academic, because these are the worlds a robot will practice in before it walks through a real kitchen with a real child in it, the worlds a surgeon's display will render, and eventually the worlds our own recorded memories get rebuilt into. Getting the geometry wrong in a film is a blooper. Getting it wrong there is something else.
So the next time a lab announces a model that builds worlds, there's one question worth asking before any of the demos: what did you hold out, and who checked?
One thing has changed that makes all of this far more practical, and it's the subject of the next post. Until recently, turning a capture into geometry took minutes of optimization, so this kind of check was always a batch job run long after the shutter closed. Reconstruction is now getting fast enough to run while the camera is still recording, which turns grading from an audit into a viewfinder.
📌 This post is part of The Spatial Shift, my personal series on AI, immersion & the future of human-centered media. Expect periodic takes on where this industry is heading — and what might come next.