PYTHON · NUMPY

"It Runs" Is Not "It Runs In Time"

Measure three costs that a passing test cannot see: whether the policy meets its deadline, whether the control tick allocates, and whether your own benchmark number is allowed to be quoted. Pass = you drive the clock into each of its three health states, explain why two of them are different failures rather than degrees of one, and make a timing measurement refuse to be quoted.

01Challenge

Your policy is correct. Every test passes. Now it has to run at a hundred hertz on a robot, and none of the things about to go wrong will produce an error message. The Bench gives you an inference-time slider and a jitter slider. Drive the clock until the fast loop has nothing to send. Then find the state where the actions keep flowing perfectly and the guarantee is nevertheless gone, and say what you would do differently in each case.

02Model

Three costs, none of which a correctness test can see. First, the deadline. A policy that infers in ninety milliseconds cannot drive a hundred-hertz loop, and the failure mode is not an exception. The fast loop asks for the next action, there isn't one, and something goes to the motors regardless. The clock in this lab has three states, and the reason there are three and not two is the whole point. NOMINAL is fine. STARVED means the queue ran dry and the stream stopped. BEYOND MARGIN means the actions are still flowing perfectly, on time, every tick - and the measured latency has exceeded the delay the plant tolerates, so the stability argument you were relying on no longer applies. Nothing looks wrong. Everything is on time. The guarantee is gone. Collapse those two into one flag and you lose the only one you could have acted on early. Notice also what gets frozen. When a new chunk arrives, some of its actions have already been sent to the motors during the inference, and those are not yours to change any more. The count is not a tuning parameter: it is the ceiling of inference time over control period. If you guess it, a new chunk contradicts a motion the robot has already made. Second, the allocations. Thirty-five heap allocations per dynamics call, inside a control tick, is a latency spike waiting for an inconvenient moment. A workspace brings it to one. But an optimisation is only free if the answers do not move, and the check here is bit-identity - the raw bit patterns, not a tolerance. Close-enough is how a reproducibility claim quietly dies, so the lab prints the maximum difference and it is zero. Third, and this one is about you rather than the machine. Drag the jitter slider. The median barely moves and the relative spread explodes, and past about a tenth the library stops calling the number a measurement and starts printing NOISY, DO NOT QUOTE. That is a benchmark refusing to let itself be repeated in a slide. Note which statistics it reports: a median and a median absolute deviation, not a mean and a standard deviation, because one paused sample moves a standard deviation and does not move these.

03Guided practice

Step A (worked, fully shown): inference 30 ms, control period 10 ms. Three cells are red: ceil(30/10) = 3 actions were consumed while the inference was in flight. Health reads NOMINAL, gaps over 400 ticks reads 0, slack reads 80 ms. This is a healthy loop. Step B (worked): drag inference to 90 ms. Latency reads 9 samples against a margin of 6, and the health word turns gold: BEYOND MARGIN. Read the gap count. It is still zero. Write down what that combination means before reading on: the robot is being commanded smoothly and continuously, and the stability guarantee has lapsed. Step C (you do it): keep going until the word turns red. Now the gap count moves. Say what the fast loop sends to the motors on a tick where the queue is dry, and why returning nothing is a better answer than repeating the last action. Step D (you do it): leave the clock and drag the jitter slider from 0 to 0.5. Watch the median and the relative spread separately, and find the jitter at which the verdict flips. Say which of the two numbers moved and which did not, and why the library reports a MAD rather than a standard deviation.

04Feedback

PASS WHEN PASS when you have reached all three health states on the live clock, stated the difference between BEYOND MARGIN and STARVED in your own words, and driven a timing measurement to NOISY. There is no code to submit.

If your run fails, check:

You reached BEYOND MARGIN and moved on because the gap count was still zero. That is precisely the state worth stopping at. Every action arrived on time and the delay margin the plant tolerates has been exceeded, so the stability argument no longer covers what you are doing. It is the only failure here you can catch before the robot shows you.
You are treating the three health states as a severity scale. They are not ordered. STARVED is a stream failure and BEYOND MARGIN is a guarantee failure, and a system can hit either without the other. Two flags, two responses.
You asked what the loop should send on a starved tick. The clock returns nothing, deliberately, because the alternative is repeating the last action - which produces plausible motion and hides the fault. A fault that looks like working is the most expensive kind.
You changed the inference time and expected the frozen count to be something you could tune. It is not a parameter. It is ceil(inference / control period), measured. If you set it smaller than the truth, the new chunk overwrites actions the motors have already executed.
You saw the allocation count go from 35 to 1 and took the win. Check the line beside it: max difference 0.0e0 and bit-identical. That is the part that makes the win real. An optimisation that changes the last bit of a dynamics result is a different simulator, and every stored trajectory that used the old one is now unreproducible.
The median barely moved when you added jitter and you concluded the jitter did not matter. Look at the relative spread instead: it went from about 0 to over 1. The median being robust is exactly why it is reported, and it is also why the median alone cannot tell you whether to trust the number.
You want to know why NOISY is a refusal rather than a warning. Because a number printed without its spread gets repeated without its spread. The verdict is inside the same line as the value so it cannot be separated from it in transit.

05Retrieve & space

Frozen actions (typed): with a 10 ms control period and a 30 ms inference, ____ actions are frozen, computed as ____ -> 3, as ceil(inference / control_period). It is a measurement, not a tuning parameter.
Two different failures (short answer): distinguish BEYOND MARGIN from STARVED -> Starved means the queue ran dry and the fast loop had nothing to send. Beyond margin means every action arrived on time but the measured latency exceeds the plant's delay margin, so the stability guarantee has lapsed while nothing looks wrong.
The honest empty answer (multiple choice): on a starved tick the clock returns nothing rather than repeating the last action. Why? -> Repeating produces plausible motion and hides the fault. A failure that looks like working is the most expensive kind.
The free optimisation (typed): the workspace path cut allocations from ____ to ____ per dynamics call, and the check that makes it safe is ____ -> 35 to 1, checked by bit-identity of the results (to_bits()), not by a tolerance.
Robust and not (short answer): adding 50% jitter moves the median about 17% and the relative spread from 0 to over 1. What does each tell you? -> The median tells you the central cost and is robust to a few bad samples; the spread tells you whether you measured the code or the machine, and only it can withdraw the number.
Why MAD (short answer): why does the benchmark report a median absolute deviation rather than a standard deviation? -> One paused sample moves a standard deviation substantially and barely moves a MAD, so the MAD describes the code where the standard deviation describes the interruption.

06Mastery & project

You separate correctness from timeliness and measure both. You know that a real-time failure has at least two independent modes and you would instrument for each. You never accept an allocation or speed optimisation without a bit-identity check on its output. And you report a spread alongside every timing number, and withdraw the number when the spread says you measured the machine.

Not a capstone lesson. It is the deployment half of Module 7: the fabric lesson shows you where a workload can run, and this one shows you what it costs to run there on time.

GO FURTHER · THE LIVE PLAYGROUND

The clock that makes `frozen` a measurement ↗ — Owns the two rates and the measured latency, keeps latency_samples uncapped so the margin check still sees an over-long inference, and returns None from tick() rather than repeating a stale action. 400 ticks, 0 gaps; zero latency reduces bit-identically to naive chunking.
Allocation-free dynamics, bit-identical ↗ — 35 allocations per forward_dynamics call became 1, verified bit-identical via to_bits(). An audit had attributed an MPPI tick's allocations to the controller; measurement found 923,730 of them and located them in core's dynamics instead.
A benchmark that refuses to be quoted ↗ — Median and MAD rather than mean and standard deviation, a trustworthy() gate, and a counting allocator that is the workspace's only unsafe block. Also where a calibration loop overflowed to usize::MAX through a saturating cast.

← Your device is a fabric The model meets reality →

PYTHON · NUMPY

"It Runs" Is Not "It Runs In Time"

Live: the chunk clock is the real ferromotion-policy ChunkClock, delivering flow-matched chunks through the real-time-chunking inpainter and ticking a 100 Hz loop; the bit-identity check runs both the allocating and the workspace dynamics path on a 6-dof arm and compares to_bits(); the timing verdict is ferromotion-bench's own Measurement. Allocation counting is the one number quoted rather than measured here, because counting needs a global allocator and installing one would tax every other lab on the page.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you