PYTHON · NUMPY

Three bands, one real number

Read the Korean three-tier sweep for what it prices, separate a converged success rate from a convergence budget, and check a reported percentage against its own stated trial count.

01Challenge

Wider bands are supposed to cost simulated performance and buy real performance. Price both halves of that trade from the published figures and find out how much of it the experiment can support.

Predict firstThe source states 65 real trials and reports 77.3 per cent and 91.7 per cent. How many whole numbers of successes out of 65 round to 77.3 per cent?

02Model

The Korean sweep trains three tiers for 10,000 episodes each and reports simulation-converged grasp success. DR1 reaches 97 per cent or better within about 1,000 steps. DR2 settles at about 94 per cent after about 3,000. DR3 reaches 86 to 89 per cent after 10,000. Ten times the training budget for 8 to 11 points less simulated success is the shape people have in mind when they say width costs something, and it is worth having in hand. The second half of the experiment is where the reading changes. There is one real endpoint on a UR16e, 77.3 per cent, and it stands for all three tiers, so differencing the two sides yields three different taxes from a single real measurement: 19.7 pp against the easiest tier and 8.7 to 11.7 pp against the hardest. The review that collected these figures then struck its own strongest Korean candidate. Both endpoints of the reported 77.3 to 91.7 per cent pair are real-world figures on the same robot, policy pool and simulator, differing only in the ranking rule; the three tiers vary object spacing and obstacle count rather than randomisation width; and the stated count of 65 real trials produces neither reported percentage. The claimed trial-budget cut from 250 to 65 reads as a 74 per cent reduction, with the 250 being 50 by 5 arithmetic rather than a run and the 65 being a calibration batch of 13 policies by 5.

03Guided practice

1. Enter the three tiers. Converged simulated grasp success against the step count at which it converges: DR1 at 97 per cent by about 1,000 steps, DR2 at about 94 per cent after about 3,000, DR3 at 86 to 89 per cent after 10,000.

2. Price the width inside simulation. Divide DR3's step count by DR1's, and difference their converged rates. Ten times the budget for less success is the trade as reported.

3. Bring in the real side. There is one real figure, 77.3 per cent on a UR16e, and it covers all three tiers. Difference it against each tier.

4. Count your axes. Print how many real endpoints and how many simulated endpoints you have. A band-against-loss curve needs at least two real points.

5. Check the denominator. Search every trial count up to 65 for a whole number of successes that rounds to 77.3 per cent, then to 91.7 per cent. Note which counts survive and whether 65 is among them.

6. Find the smallest count that could produce both. Extend the search and read off the number.

04Feedback

PASS WHEN PASS when cell 1 prints a 10x step ratio for 8 to 11 pp less converged success, cell 2 prints taxes of 19.7, 16.7 and 8.7 to 11.7 pp with a spread of 11.0 pp against a smallest tax of 8.7 pp, and cell 3 prints a 74.0 per cent claimed cut, 22 and 44 as the only counts up to 65 producing 77.3 per cent, multiples of 12 for 91.7 per cent, False for both 65 checks, and 132 as the smallest count producing both.

If your run fails, check:

If your denominator search returns nothing at all, widen the tolerance to 0.05 percentage points. A value reported to one decimal place is any true value within half of the last digit.
If it returns dozens of counts, your tolerance is too wide. At 0.05 pp the window is 0.001 in fraction terms, and for counts up to 65 the spacing between neighbouring fractions is at least 1 in 65, so at most one numerator can fit per count.
The spread in cell 2 is the finding. Three taxes from one real measurement means the number you would quote is set by which simulated tier you choose to difference against, and the tiers vary scene difficulty rather than band width.

05Retrieve & space

How many steps does DR1 need to pass 97 per cent, and how many does DR3 need to reach 86 to 89 per cent?
How many real endpoints does the three-tier sweep report, and how many tiers do they cover?
What do the DR1, DR2 and DR3 tiers actually vary between them?
Why does a stated count of 65 real trials fail to support either 77.3 or 91.7 per cent?
What does the claimed trial-budget cut from 250 to 65 come to as a percentage, and what did the review find the 250 to be?

06Mastery & project

You can read a band sweep for what it prices, tell a converged number from a convergence budget, and check a reported percentage against its stated denominator before you cite it.

Take the last real-robot percentage you saw quoted anywhere. Work out which trial counts could produce it to the decimal place given. If the stated trial count is not among them, record that beside the figure before you cite it, and record what the figure would need to be believed.

GO FURTHER · THE LIVE PLAYGROUND

Sim to Real: The Road to Physical Agency (TR-2026-35) ↗ · Section 5 for the three tiers and the single real endpoint, and the struck-claims table for why the review removed its own strongest Korean transfer figure.
Unified Sim-and-Real Scoring Methods for Robot Policy Selection, JKROS 20(3):371-380 ↗ · The primary source, in Korean. Section 4.3 and Table 3 hold the tier definitions, which is where object spacing and obstacle count turn out to be what varies.
RoboChallenge: large-scale real-robot evaluation of embodied policies ↗ · The variance term every ranking in this material has to survive. With task, props and model held constant, a measured real success rate moves from 0 to 100 per cent depending on which class of human resets the scene.

← Four bands and a missing axis The contact limit is a published field →

PYTHON · NUMPY

Three bands, one real number

Price the trade, count the axes, then check the arithmetic of the trial budget. Standard library only.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you