PYTHON · NUMPY

What a bench hour buys

A paper reports 1 to 2.5 hours of real-world training per task, and its appendix gives the transitions collected and a 10 Hz environment update rate. What will the ratio of experience time to reported time turn out to be?

Recover the duty cycle of a published real-robot training run from its own appendix, and say how much of a reported training hour was the robot gathering experience.

01Challenge

The most sample-efficient real-robot RL result published reaches 100% success in 1 to 2.5 hours of training. Its appendix publishes, per task, the number of transitions collected and the rate they were collected at. Work out how much of that training time the robot was actually moving under the policy, and budget a bench from the answer rather than from the headline.

Predict firstA paper reports 1 to 2.5 hours of real-world training per task, and its appendix gives the transitions collected and a 10 Hz environment update rate. What will the ratio of experience time to reported time turn out to be?

02Model

This course has taught you to collect demonstrations, to clone a policy, to correct it on the robot and to ship it. The number everyone quotes from the most sample-efficient real-robot result published is a wall clock: one to two and a half hours of training per task, reaching a hundred percent success. It is a real measurement and it is not a lie. It is also not the number you can budget a bench from, and the paper hands you everything needed to see why. Its appendix publishes, for each task, the total number of reinforcement learning transitions collected and the environment update frequency, ten hertz. So one transition is one tenth of a second of robot motion under the policy, and the transitions convert exactly into seconds of experience. Divide that by the training hour the paper printed beside it, and you have the duty cycle of a real robot learning: the share of bench time that was actually the robot gathering experience rather than being reset, repositioned, or waited on. Section four point three even licenses the division by saying the training time includes all scripted motion, policy rollouts, intended stops and onboard computation. What it never does is divide. You will, and then you will check whether the answer survives the one place the appendix is ambiguous.

03Guided practice

1. Run the gate before you trust the table. The first cell makes the hand-typed table reproduce three numbers the paper printed in its own Average row. If those fail, stop: the typing is wrong. 2. One transition is 1/hz seconds. At 10 Hz a transition is 0.1 s of robot motion, so transitions divided by hz is experience in seconds. The denominator is the paper's Training Time column, in hours, so multiply by 3600. 3. Sum over sum, not the mean of the ratios. The tasks run for one hour to six hours. Averaging the per-task fractions weights a one-hour task like the six-hour one, and gives a different answer. The cell prints both so you can see the gap. 4. Then read the ambiguity. Eleven appendix tables cover twelve trained tasks, because one table serves both IKEA side panels. The last block charges its transitions to each side panel in turn and shows the answer barely moves, which is what makes the finding safe to publish.

04Feedback

PASS WHEN You recovered a duty cycle of 51.53% from a paper that published every input and never divided, and you showed the answer survives the one place its appendix is ambiguous.

If your run fails, check:

If your fraction is near 0.52 but not 0.5153, you averaged the per-task ratios. Sum the experience and sum the wall clock, then divide the totals.
The Training Time column is in hours and the transitions convert to seconds. One of the two needs a factor of 3600.
Use only the tasks present in the transitions dict; two tasks in the table deliberately have no unambiguous figure.

05Retrieve & space

The paper states in its own text that the training time includes scripted motion, rollouts, stops and computation. Given that, why is the duty cycle a finding rather than a criticism?
The mean of the per-task fractions and the sum-over-sum differ. Say which one a person buying bench time should use, and what the other one would be right for.
Eleven appendix tables cover twelve trained tasks. Describe what you did about that, and what you would have had to do if the answer had moved by twenty points instead of half a point.

06Mastery & project

You can turn a reported wall clock into the experience it actually contained, using only figures the authors published, and state the ambiguity in your own answer with the sensitivity attached rather than hidden.

Budget a bench for a task of your own. Write the hours you would book, the duty cycle you expect, and therefore the experience you would actually buy. Then write the one change to your rig that would move the duty cycle most, and say whether it is a change to the policy or to the reset.

GO FURTHER · THE LIVE PLAYGROUND

The result ↗ · Luo, Xu, Wu and Levine: human-in-the-loop RL reaching 100% on precise contact-rich tasks in 1 to 2.5 hours per task, with the per-task transitions and the 10 Hz update rate in its appendix.
The on-robot flywheel ↗ · The loop this lesson prices: the same cycle, counted in bench hours rather than in improvement.
PAI-330 · The Cost of the Loop ↗ · Where the half-hour that is not experience gets converted into joules, because a robot holding position still draws power.

← Capstone: Build the Whole Pipeline Back to course →

PYTHON · NUMPY

What a bench hour buys

One published table, one appendix figure, and the half of every bench hour the headline does not mention.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you