PAI-190 · Education

World Models: Learn a Simulator You Can Imagine In

Build a world model, a learned simulator that predicts what happens next, from the ground up. You fit the smallest one to a real system, roll it forward to imagine the future and watch error compound, make it nonlinear and latent the way Dreamer and DIAMOND do, then plan with it and finally play a pixel-space world the way Genie 3 does. It ends where the 2026 frontier actually is: Meta's V-JEPA 2 (physics learned from a million hours of video, then robot control from a little real data) and NVIDIA's Cosmos world-foundation models, the same imagine-then-act loop the Institute's own runs live and on-device. And it meets that frontier's honest limits: benchmarks like PhysicsMind show today's world models lean on superficial cues and routinely violate balance, torque, and inertia, which is why structured, physics-grounded models (port-Hamiltonian dynamics, PH-Dreamer) are the live research direction; and a rollout's score is not proof it is safe to run for real, which is why runtime verification of an imagined plan, before you commit it, is being built now, ours included.

Skilled·4 modules·7 lessons
▶ Start the course ← All courses
Module 1

Predict the next state

A world model is next-state prediction learned from data; fit the smallest one and roll it forward.

  1. L2Predict the next stateMeet the world model in one line: a map from (state, action) to the next state, learned from experience.→
  2. L2Imagine a trajectoryRoll the model on its own predictions to imagine the future — and watch prediction error compound.→
Module 2

Learn to imagine

Make the model nonlinear and latent — the representations modern world models are built on.

  1. L3A model that bendsGive the world model a nonlinear basis so it can fit the curved parts of physics a straight line misses.→
  2. L3Imagine in a latentCompress a high-dimensional observation to a small latent, then imagine the future inside it — the core trick of modern world models.→
Module 3

Imagine, then act

Turn a world model into a controller and into a playable world.

  1. L4Plan by imaginingUse the world model to choose actions — search plans in imagination, commit only the best to the real world.→
  2. L4A playable worldBuild a pixel-space, action-conditioned world model and play it — the exact principle behind Genie, Oasis, and our on-device world model.→
Module 4

Certify before you commit

Contrast scoring an imagined plan by its predicted outcome with certifying it never left what the model actually knows — the gap between every current sample-and-score gate and a formal certificate-before-commit.

  1. L4Certify, don't just scoreSee the difference between trusting a plan because the model scored it well, and trusting it because it was certified to stay inside what the model actually knows.→
See it live

Anatomy demonstrations

The machines behind this course, taken apart three ways — the body, the one rule, and the small learned brain. Guess before you look; an open core proves every number on the page.

DemonstrationThe learned brainsOn-device policies you can watch decide →The whole series →
Foundations

From the interactive textbook

The ideas under this course as live explorables — each runs the real Rust library and re-derives its own result.

Ch. 9 · ModelingMake it linearOpen the chapter ↗The full textbook →