PYTHON · NUMPY

What imagination costs

A pipeline is made 38x faster. One of its steps distributes the work across two GPUs for a 47% latency cut. What is the energy saving per action chunk?

Decompose a published inference-optimisation ladder into the rungs that deleted work and the rungs that bought hardware, and price each in energy rather than in latency.

01Challenge

A 14B world-action model goes from 5.7 seconds per action chunk to 150 ms, a 38x speedup, and closes a 7 Hz loop on two GB200s. Work out how much of that 38x came from doing less arithmetic, and what the speedup is worth once you count joules instead of seconds.

Predict firstA pipeline is made 38x faster. One of its steps distributes the work across two GPUs for a 47% latency cut. What is the energy saving per action chunk?

02Model

You have built a world model, planned in it, and certified a plan before committing it. None of that asked what imagining costs, and for a model of this size it is the question that decides whether it can be a controller at all. DreamZero is a fourteen billion parameter world-action model that acts by predicting future frames and the actions that produce them, and what is unusual about the paper is not the model, it is that the inference section publishes its whole optimisation ladder. A naive single-GPU implementation needs about five point seven seconds per action chunk. The deployed system needs a hundred and fifty milliseconds and closes the loop at seven hertz on two GB200s, which the paper reports as a thirty-eight times speedup. A speedup table like that invites you to add up the factors and stop, and the interesting work starts one question later: what did each rung spend. Some rungs delete arithmetic, which is free in every sense. One rung distributes the work across a second GPU, which buys time with power. And because energy is power multiplied by time, those two kinds of rung are not the same kind of win at all. You will separate them, carry the energy through the ladder, and find that the headline number and the energy number differ by a factor you can name in advance.

03Guided practice

1. Energy is power times time. One line: the time shrank by latency_factor and the power grew by power_factor, so the energy changed by their quotient. Check yourself on twice as fast at the same power, which is 0.5. 2. Watch for the rung above one. Three rungs will come out below 1.0 and one will not. The one that does not is the only one that bought hardware. 3. The last rung is inferred, not published. The paper gives a total and three of its steps, so the fourth is the total divided by the product of the three. Note that this makes it a residual, and a residual carries whatever the paper did not itemise. 4. Shares of a product are shares of its logarithm. The rungs multiply rather than add, so attributing the 38x means comparing logarithms. Adding the factors would give you a number with no meaning.

04Feedback

PASS WHEN You separated a speedup into the rungs that deleted arithmetic and the one that bought a second GPU, and found that the 38x headline is a 19x energy saving, with the factor of two being the hardware itself.

If your run fails, check:

If every rung comes out below 1.0, the power factor is not reaching your calculation. One rung draws twice the power.
Energy is power times time, so it is a quotient and not a product of the two factors. Twice as fast at twice the power changes nothing.
Define energy_factor(latency_factor, power_factor) returning one number.

05Retrieve & space

Three rungs cut energy and one raised it. State the property that separates them, without naming the rungs.
The final rung's factor was inferred from the total rather than published. Say what that residual could be hiding, and why it is still worth computing.
The speedup and the energy saving differed by exactly 2.0. Predict what that factor would be for a system distributed over eight GPUs, and say what would have to be true for it to be smaller.

06Mastery & project

You can read a published optimisation ladder as two ladders, one in seconds and one in joules, say which rungs deleted work and which bought hardware, and predict the divergence between a speedup and its energy saving from the hardware count alone.

Take any speedup you have achieved or read about, and split it the same way. For each step, write the latency factor, the power factor, and the quotient. Then state the speedup and the energy saving as two separate numbers. If a step raised the power and you cannot say by how much, that step's energy is unknown and so is the total, which is itself the finding.

GO FURTHER · THE LIVE PLAYGROUND

The model ↗ · Ye, Ge, Zheng et al., NVIDIA: a 14B world-action model that acts zero-shot by predicting frames and actions, with its inference optimisation ladder published in full.
Which block was that number measured over? ↗ · The companion question asked of an efficiency figure rather than of a speedup.
PAI-330 · The Cost of the Loop ↗ · Where this accounting becomes the whole subject, including what a total hides and what a receipt should refuse.

← Certify, don't just score Back to course →

PYTHON · NUMPY

What imagination costs

One published speedup ladder, four rungs, and the difference between buying time and saving energy.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you