PYTHON · NUMPY · IN-BROWSER

Did It Learn, or Memorize?

Tell generalization from memorization by comparing training error to held-out error, and see why normalization is what buys the difference.

01Challenge

With un-normalized inputs the policy memorizes and its held-out error stays high. Turn normalization on and re-run until held-out error is small.

02Model

A tiny training error means nothing on its own; a lookup table has zero training error and learns nothing. The honest signal is the held-out error, on episodes the policy never saw. When training error is tiny but held-out error is large, you memorized. Here the fix is the same trick from lesson two: normalize. Without it, the different input scales make training unstable and the policy overfits the frames it saw; with it, held-out error drops to meet the training error, and you have a policy that generalizes.

03Guided practice

1. Two numbers. Train, then measure error on the training frames and on the held-out episodes.

2. Read the ratio. If held-out error is many times the training error, the policy memorized.

3. Flip normalization on. Set NORM=True and re-run. Normalized inputs train stably, and held-out error falls to meet training error — the mark of a policy that learned the task.

04Feedback

PASS WHEN PASS when held-out error is small and close to the training error — the policy generalizes rather than memorizes.

Held-out error is still high — set NORM=True and re-run.
A cell errored — run them top to bottom.

05Retrieve & space

Why is a small training error not enough? (a memorizer has zero training error and no skill).
What is the honest measure of a policy? (its error on held-out episodes).
What buys generalization here? (normalizing the inputs so training is stable).

06Mastery & project

You can diagnose memorization versus generalization and know the preparation that separates them.

The check that keeps you honest on every model you ship: hold out real episodes, and trust the held-out number.

← When Cloning Drifts: DAgger Crossing the Reality Gap →

PYTHON · NUMPY · IN-BROWSER

Did It Learn, or Memorize?

Compare training and held-out error to diagnose memorization, then fix it with normalization.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you