PAI-160 · Education

Robot Learning: From Demonstrations to a Released Policy

Teach a robot to reach by showing it, not programming it. You turn demonstrations into data, clone the behavior with a linear policy and then a small neural network trained by gradient descent, learn the honest test that separates generalizing from memorizing, and finish by running the Institute's released checkpoint — the exact forward pass — before watching the real weights drive the arm live. Every lesson runs in your browser, on your device.

Intermediate → Skilled·3 modules·9 lessons
▶ Start the course ← All courses
Module 1

The data is the program

Frame control as demonstrations, and prepare a set a policy can actually learn from.

  1. L2Program vs. PolicySee why a hand-written controller falls short on a reach task, and why the data itself becomes the program.→
  2. L2The Dataset Is the ProgramPrepare a demonstration set the way a policy needs it: normalized inputs and an honest, episode-level split.→
Module 2

Behavior cloning

Clone the expert with a linear policy, then a neural one trained by gradient descent.

  1. L3Linear Behavior CloningFit the simplest possible policy — one linear map from observation to action — in closed form, and measure it on held-out episodes.→
  2. L4A Neural Policy, Trained by Gradient DescentBuild a small multi-layer policy with the same shape as the released checkpoint, and train it by gradient descent until it beats the linear floor.→
  3. L4When Cloning Drifts: DAggerSee why a cloned policy drifts on the states it visits, and fix it with DAgger — training the policy on its own distribution.→
Module 3

Generalize, ship, and make your own

Prove the policy generalizes, then run the released checkpoint yourself.

  1. L4Did It Learn, or Memorize?Tell generalization from memorization by comparing training error to held-out error, and see why normalization is what buys the difference.→
  2. L4Crossing the Reality GapSee why a policy that is perfect in simulation can fail on real hardware, and make it robust with domain randomization.→
  3. L4Run the Released CheckpointRun the exact forward pass of the released checkpoint, understand its 17,923 parameters, and hand off to the live demo.→
  4. L4Capstone: Build the Whole PipelineAssemble everything — normalize, split by episode, train, and verify on held-out episodes — into one shippable policy, then go make your own in Forge.→