PYTHON · NUMPY

Program vs. Policy

You will tune the single gain K of a hand-written controller to copy an expert reacher. How close can one fixed gain get to the expert?

See why a hand-written controller falls short on a reach task, and why the data itself becomes the program.

01Challenge

A hand-coded controller uses one fixed gain on the target error. Tune the gain K to make it as good as you can, then read the gap to the expert it is trying to copy.

Predict firstYou will tune the single gain K of a hand-written controller to copy an expert reacher. How close can one fixed gain get to the expert?

02Model

The old way to move a robot is to write the rule yourself: if the target is to the right, turn right. It works until the task has any subtlety, an obstacle to curve around, joints that couple. The new way is to show the robot what to do and let it fit the rule from the demonstrations. This is imitation learning, and its program is a table of what the robot saw and what an expert did. You will feel the ceiling of hand-coding here, so the rest of the course has something to beat.

03Guided practice

1. The data. Each row is one control tick: a 7-number observation (three joint angles, the target offset, the obstacle offset) paired with the expert's 3-number joint move.

2. The hand rule. It only looks at the target error and ignores the joints and the obstacle, so it can never match the expert exactly. Raising K helps up to a point, then overshoots.

3. The lesson. No single gain wins. That gap is the room a learned policy will fill.

04Feedback

PASS WHEN PASS once the cells run: you have the observation→action data and a baseline error to beat.

If your run fails, check:

Read the bench’s printed result, it passes when the cells run: you have the observation→action data and a baseline error to beat. Run the cells top to bottom; if one errors, the FAIL line names the step to fix.
Try a few values of K and watch the hand-coded MSE move.

05Retrieve & space

What are the two halves of one row of imitation-learning data? (an observation and the expert's action).
Why can't the hand rule match the expert? (it ignores the joints and the obstacle).
What does the mean-baseline MSE represent? (the error of always predicting the average action).

06Mastery & project

You can frame a control task as observation→action data and state why hand-coding hits a ceiling.

The habit the course is built on: before writing a controller, ask whether the rule should be learned from demonstrations instead.

← Robot Learning: From Demonstrations to a Released Policy The Dataset Is the Program →

PYTHON · NUMPY

Program vs. Policy

Load the reach task as data, then hand-code a controller and measure how far it falls short of the expert.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you