PYTHON · NUMPY

What generating an answer costs

The same 200-token answer on an H100 at FP8 versus an older V100. How far apart?

Put a joule figure on one generated answer, from published numbers rather than a vendor ratio.

01Challenge

A vendor says its typed model is a hundred times cheaper than generating. Before you can check that, you need the thing it is a hundred times cheaper THAN. Compute it.

Predict firstThe same 200-token answer on an H100 at FP8 versus an older V100. How far apart?

02Model

Energy per answer is not a mystery: it is tokens times joules per token, and both halves are published. The spread is wide and honest, 0.39 J per token for LLaMA3-70B on an H100 at FP8, around 1.8 at a mid-range estimate, 3 to 4 on older V100 silicon. A 2026 peer-reviewed analysis in Joule found widely cited per-query figures overstated by four to twenty times against production-optimised deployments, so which number you pick decides the argument before it starts. Pick the measured one, name it, and say so.

03Guided practice

Multiply tokens by joules per token. Then swap in the V100 figure and watch the baseline move nine-fold. That sensitivity is why a ratio quoted without its baseline tells you nothing.

04Feedback

PASS WHEN You produced 78.0 J at the measured figure and showed the identical answer costs nine times that on older silicon.

If your run fails, check:

A ratio without its baseline is not a measurement.
Hardware moves the baseline further than most optimisations move the ratio.

05Retrieve & space

Why does a cost ratio mean nothing until its baseline is named?
Two teams report the same model, one at 78 J per answer and one at 700 J. What could both be right about?

06Mastery & project

You can state the energy of one generated answer with its hardware, precision and token count attached.

Take a task your own system generates an answer for. Count its output tokens and state its baseline in joules, with the per-token figure you used and where it came from.

GO FURTHER · THE LIVE PLAYGROUND

What a decision costs: the instrument ↗ · Move the router and watch the ceiling move with it.

← What a Decision Costs: Python to Measure It, Rust to Move It The miss pays twice →

PYTHON · NUMPY

What generating an answer costs

Three published per-token figures, one answer length. Compute the baseline the rest of the course compares against.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you