PYTHON · NUMPY

Energy at one length, accuracy at another

An analog system is 3.6x more energy efficient but loses accuracy to noise; four sampled answers buy the accuracy back. What is left of the advantage?

Combine a published energy advantage with the accuracy it costs, find that the two were measured on different workloads, and locate the single unreported quantity that decides whether the advantage survives.

01Challenge

A NeurIPS 2025 paper shows an analog in-memory computer running a language model 3.6 times more efficiently than an idealised GPU, and shows that the analog noise costs accuracy which can be bought back by sampling several answers. Work out what is left of the 3.6x once the accuracy has been bought back, and then find out why the paper cannot tell you.

Predict firstAn analog system is 3.6x more energy efficient but loses accuracy to noise; four sampled answers buy the accuracy back. What is left of the advantage?

02Model

The last lesson recovered a denominator a table did not print. This one is harder, because two honest numbers from the same paper cannot be combined at all without a third that it never gives. The paper trains a language model to tolerate the noise of analog in-memory computing, where arithmetic is done by currents flowing through a memory array instead of by moving data to a processor. It reports one energy figure: in simulation, 3.6 times the efficiency of a GPU idealised to run at full utilisation with no cost for moving data. And it reports what the noise does to accuracy on competition mathematics: the analog model gets fewer problems right, but if you let it answer several times and vote, it catches up. Both findings are careful and real. The trouble starts when you ask the obvious question, which is how much of the 3.6x is left after you pay for those extra answers. Extra answers share the prompt, so only their generation is repeated, and that means the cost depends on how long an answer is compared with its prompt. The energy was simulated on answers 64 tokens long, because the simulator could not handle longer ones. The accuracy was measured on answers allowed up to 2,048 tokens. You are going to find that the verdict flips between those two lengths.

03Guided practice

1. Read the parity table first. Two answers leave the analog model 3.36 points short and four put it past the digital one. Three was not measured, so four is the honest count. 2. The prompt is paid for once. n answers to one prompt cost prefill_fraction + n * (1 - prefill_fraction) in units of one answer. With no prefill that is n; with all prefill it is 1. 3. Divide the efficiency by that cost. Check yourself at one sample, which must return the efficiency unchanged whatever the split. 4. Then read the two ends of the ruler. The last block evaluates the net advantage at the answer length the energy was measured on and at the cap the accuracy was measured under, and solves for the length where it crosses 1.

04Feedback

PASS WHEN You combined an energy advantage with the accuracy it costs, and showed the verdict runs from a 2.25x win at the length the energy was measured on to a small loss at the length the accuracy was measured under, crossing at 1,664 generated tokens.

If your run fails, check:

If one sample does not return the efficiency unchanged, the prefill is being charged more than once.
If four samples always cost exactly four, you are ignoring that the prompt is shared. Prefill is paid once.
Define net_advantage(efficiency, n_samples, prefill_fraction) returning one number.

05Retrieve & space

Both of the paper's numbers are careful and correct. Say precisely why they cannot be multiplied together.
The simulator could only run 320-token sequences. Explain why that limitation matters more for this question than the idealised GPU does.
Name the one measurement that would settle the verdict, and say which way you expect it to come out for competition mathematics and why.

06Mastery & project

You can take an efficiency figure and an accuracy figure from the same work, check whether they were measured on the same workload before combining them, and when they were not, find the quantity that decides the combined answer and state the verdict on each side of it.

Find a paper, datasheet or product claim that reports both an efficiency advantage and an accuracy cost. Write down the workload each was measured on: input length, output length, batch and task. If they differ, write down the one quantity you would need to combine them, and whether the source reports it.

GO FURTHER · THE LIVE PLAYGROUND

The paper ↗ · Analog Foundation Models, NeurIPS 2025: analog-noise-aware training for language models, with the energy estimate in Appendix K and the sampling results in Table 20.
What the table would not say ↗ · The previous lesson: a missing denominator that physics could recover. This one cannot be recovered, only bounded.
PAI-310 · What a factor describes ↗ · Nine efficiency factors sorted by what was measured and in what denominator. This lesson adds the one column that table does not have: what the factor costs in accuracy.

← What the table would not say Back to course →

PYTHON · NUMPY

Energy at one length, accuracy at another

Two careful numbers from one paper, measured on two different workloads, and the unreported length that decides whether they add up.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you