PYTHON · NUMPY

Which data, not how much

You can collect 1600 demonstrations. Is it better to take 400 each in 4 environment-object pairs, or 50 each in 32 pairs?

Allocate a fixed demonstration budget using the published data scaling law, and show that the intuitive allocation has an error floor no budget can cross.

01Challenge

You have one afternoon and four people, which is about 1600 demonstrations. The instinct is to pick a few setups and collect deeply in each. Price that instinct against the measured scaling law, then say what an unlimited budget would buy you if you kept the same few setups.

Predict firstYou can collect 1600 demonstrations. Is it better to take 400 each in 4 environment-object pairs, or 50 each in 32 pairs?

02Model

The previous lesson said the dataset is the program. This one says which dataset. A 2025 study collected over forty thousand demonstrations and ran over fifteen thousand real rollouts to answer the question every student asks first, which is how much data, and the answer turned out to be that how much is the wrong question. Generalisation to unseen environments and unseen objects follows a power law in the number of distinct environments and the number of distinct objects, and the exponent on objects is the steepest thing they measured. Demonstrations per condition behave completely differently: they help up to a point and then stop. Their own numbers give that point away. They report that eight pairs plateau at four hundred demonstrations, sixteen at eight hundred, and thirty-two at sixteen hundred, and when you divide each of those you get fifty every time. The plateau is not a property of the dataset, it is a property of one environment-object pair, so collecting past it is collecting nothing. That turns a vague instinct into arithmetic you can do before you start, and it changes the answer.

03Guided practice

1. Read the plateau rows first. Divide each published plateau by its pair count before you do anything else. Three rows, one number. 2. Whole pairs only. budget // per_pair, because half an environment-object pair is not a thing you can collect. 3. The cap is per pair, not per dataset. min(per_pair, K) is the whole lesson: a pair cannot absorb more than K, no matter how large the budget is. 4. Then read the floor. The last block holds the pair count fixed and lets the budget go to infinity. Notice that the error does not move, because every demonstration past 50 * pairs is in the wasted column.

04Feedback

PASS WHEN You priced the instinct: the same 1600 demonstrations spread over 32 pairs instead of 4 wastes none of the afternoon and lands 2.114x lower error, and no budget rescues the narrow plan.

If your run fails, check:

If your wasted column is always zero, the cap is on the wrong quantity. The plateau limits what one PAIR absorbs, not what the dataset holds.
Use floor division for pairs. A budget of 1600 at 300 each buys 5 pairs, not 5.33.
Define plan(budget, per_pair) returning exactly three numbers: pairs, useful, wasted.

05Retrieve & space

The plateau was published as 400, 800 and 1600 demonstrations. Why is it wrong to read any of those three as the amount of data a task needs?
The exponent on objects is -0.49 and on environments -0.27 for the same task. You have one afternoon. Where do you spend it, and what does the difference in exponent actually buy?
State the one claim in this lesson that an unlimited budget cannot overturn, and say why the budget is irrelevant to it.

06Mastery & project

You can turn a published scaling exponent into a data-collection plan, say which axis of a dataset a budget should buy, and recognise an allocation whose error is bounded below no matter what it spends.

Write the collection plan for a task you care about before collecting anything. State the number of distinct environments, the number of distinct objects, the demonstrations per pair, and the total. Then state which of those four numbers you would raise first if the afternoon turned into a week, and which you would not raise at all.

GO FURTHER · THE LIVE PLAYGROUND

The study ↗ · Lin, Hu, Sheng, Wen, You and Gao, ICLR 2025: over 40,000 demonstrations and over 15,000 real-world rollouts, with the fitted exponents in Figure 5.
The dataset is the program ↗ · The lesson this one answers: if the data is the program, this is which program to write.
How many seeds ↗ · The same kind of question asked of the evaluation rather than of the collection.

← The Dataset Is the Program Linear Behavior Cloning →

PYTHON · NUMPY

Which data, not how much

One published scaling law, one afternoon's budget, and the allocation that wastes seven eighths of it.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you