PYTHON · NUMPY

The Dataset Is the Program

You split a demonstration set into train and test. Splitting by frame is easier than splitting by episode. What goes wrong?

Prepare a demonstration set the way a policy needs it: normalized inputs and a strict, episode-level split.

01Challenge

Compute the normalization stats and split the data by whole episodes, not by frame, so the held-out set is a fair test.

Predict firstYou split a demonstration set into train and test. Splitting by frame is easier than splitting by episode. What goes wrong?

02Model

A learned policy is only as good as the set it learns from, and two preparation choices decide whether it will ever generalize. First, normalization: put every input on the same scale so no single number dominates the gradient. Our released checkpoint stores exactly these mean and standard-deviation numbers in its config. Second, the split: if you shuffle frames and hold out a random 30 percent, neighbouring frames from the same episode leak the answer, and your test score is a lie. Hold out whole episodes instead.

03Guided practice

1. Normalize. Subtract the mean and divide by the standard deviation of each observation column. After this, every column has mean 0 and standard deviation 1.

2. Fit on train only. In the next lesson you will compute these stats on the training split alone, never on the held-out episodes.

3. Split by episode. Assign each frame to an episode, then hold out a couple of whole episodes. That is the only split that measures generalization.

04Feedback

PASS WHEN PASS when the data is z-scored (each column mean 0, std 1) and split into non-empty train and held-out sets by whole episodes.

If your run fails, check:

A cell errored, run them in order.
Check that both the train and held-out masks contain frames.

05Retrieve & space

Why normalize the observations? (so no single input dominates the gradient).
Why split by episode instead of by frame? (adjacent frames leak the answer, inflating the score).
On which split do you fit the normalization stats? (the training split only).

06Mastery & project

You can turn raw demonstrations into a normalized, strictly-split dataset ready for training.

The discipline behind every trustworthy result in the course: fit stats on train only, and test on held-out episodes.

← Program vs. Policy Linear Behavior Cloning →

PYTHON · NUMPY

The Dataset Is the Program

Compute normalization statistics and build an episode-level train/held-out split.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you