Robot Learning: From Demonstrations to a Released Policy
Teach a robot to reach by showing it, not programming it. You turn demonstrations into data, clone the behavior with a linear policy and then a small neural network trained by gradient descent, learn the test that separates generalizing from memorizing, and finish by running the Institute's released checkpoint, the exact forward pass, before watching the real weights drive the arm live. Every lesson runs on your device.
▶ Start the course ← All coursesThe credential the ecosystem leaves on the table.
The $100 SO-101 arm, Hugging Face's LeRobot, and the worldwide hackathons made teleoperate → record → train → deploy something anyone can do. What they do not give you is a credential, the LeRobot course issues none, by design. This course credentials the same skills, assessed on your own runs: pass the labs and claim a verifiable, standards-aligned certificate you can take anywhere. We certify what you demonstrate here, not a course we did not run.
Where this sits, and what moves it.
Binding constraint · How much a demonstration covers. A cloned policy is exact where it was shown and extrapolating everywhere else, and the boundary between those is not visible from the training error.
Teaching a robot by showing it required a research lab: instrumented teleoperation, a data pipeline, and a training cluster. The arc from demonstration to deployed policy was not something a student could run end to end.
A hundred-dollar arm and open tooling put the whole arc in reach, and this course runs it. The part that has not been solved by cheaper hardware is the test: separating a policy that generalises from one that memorised is still where most published results are weakest, and it is the rung this course spends the most time on.
What changes the field is a generalisation test as standard as a held-out set is in vision -- one that measures coverage of the state space a body will actually enter, not of the states it happened to be shown. That is a measurement problem before it is a learning problem, which is why it is tractable now.
Every hard thing was impossible until the constraint that made it impossible was named. How we read a frontier →
The data is the program
Frame control as demonstrations, and prepare a set a policy can actually learn from.
- L2Program vs. PolicyYou will tune the single gain K of a hand-written controller to copy an expert reacher. How close can one fixed gain get to the expert?See why a hand-written controller falls short on a reach task, and why the data itself becomes the program.→
- L2The Dataset Is the ProgramYou split a demonstration set into train and test. Splitting by frame is easier than splitting by episode. What goes wrong?Prepare a demonstration set the way a policy needs it: normalized inputs and a strict, episode-level split.→
Behavior cloning
Clone the expert with a linear policy, then a neural one trained by gradient descent.
- L3Linear Behavior CloningYou fit a single linear map from observation to action in closed form. How does it do against predicting the mean action?Fit the simplest possible policy, one linear map from observation to action, in closed form, and measure it on held-out episodes.→
- L4A Neural Policy, Trained by Gradient DescentYour multi-layer policy trains but lands short of the target. The architecture matches the released checkpoint. What is the first thing to change?Build a small multi-layer policy with the same shape as the released checkpoint, and train it by gradient descent until it beats the linear floor.→
- L4When Cloning Drifts: DAggerA behaviour-cloned policy drifts badly when it starts somewhere the expert never demonstrated. What actually fixes it?See why a cloned policy drifts on the states it visits, and fix it with DAgger, training the policy on its own distribution.→
Generalize, ship, and make your own
Prove the policy generalizes, then run the released checkpoint yourself.
- L4Did It Learn, or Memorize?Your policy nails the training set but its HELD-OUT error stays high, with un-normalized inputs. What is happening, and the fix?Tell generalization from memorization by comparing training error to held-out error, and see why normalization is what buys the difference.→
- L4Crossing the Reality GapA policy is perfect in simulation and brittle on hardware. You train it across gains from 0.5 to 2.5 instead of one nominal value. What does that buy?See why a policy that is perfect in simulation can fail on real hardware, and make it robust with domain randomization.→
- L4Beyond Frozen: The On-Robot FlywheelA deployed policy keeps learning from what it meets. You gate each update on the worst case rather than the average. What changes?See that 2026's best robot policies keep learning after deployment, and why gating an update on the worst case, not the average, is the harder problem this course has been building toward.→
- L4Run the Released CheckpointThe released checkpoint has 17,923 parameters. What does running its exact forward pass demonstrate?Run the exact forward pass of the released checkpoint, understand its 17,923 parameters, and hand off to the live demo.→
- L4How Many Seeds Does a Ranking Need?You run one seed and policy A beats policy B on every task, all signs correct. How much evidence is that?Compute how much evidence a claim like 'policy A beats policy B' actually requires, and what a multi-task suite does to that number. Pass = you derive the seed count for a 0.8-versus-0.7 comparison, confirm it empirically, and state the cost of comparing 31 tasks at once.→
- L4Capstone: Build the Whole PipelineYour pipeline lands short of 98 percent. You have normalisation off and the learning rate low. Which do you fix?Assemble everything (normalize, split by episode, train, and verify on held-out episodes) into one shippable policy, then go make your own in Forge.→
Anatomy demonstrations
The machines behind this course, taken apart three ways, the body, the one rule, and the small learned brain. Guess before you look; an open core proves every number on the page.
From the interactive textbook
The ideas under this course as live explorables, each runs the real Rust library and re-derives its own result.