PYTHON · NUMPY

Why the multiply-free win is a memory win

On real edge silicon, show that a naive ternary decode is stalled by a branchy unpack, then replace the select-chain with a branch-free arithmetic decode v=(c^2)-2 — bit-identical, and the measured decode kernel jumps from 2.6x to 5.8x faster than bf16.

01Challenge

The honest joules lesson left you with a verdict: once the multiplies are gone, the win is capped by moving bits, not multiplying them. So the multiply-free kernel should run at the memory-ratio limit — 2-bit weights are 8x smaller than bf16, so ~8x faster, right? Run it on a real GPU and the first kernel gives ~2.6x, not 8x. Something is eating five-eighths of the win. This lesson finds it, names it, and deletes it — and the fix is two integer operations you can write in one line and prove bit-exact.

Predict firstTernary weights pack to 2 bits — 8x smaller than bf16. A first, correct multiply-free decode kernel on a real edge GPU therefore runs about how much faster than bf16?

02Model

This is the systems lesson where the frontier thesis meets a real GPU, and the honest joules get honest names. You packed ternary weights at 2 bits per trit — the memory win from the last lesson, 8x less data than bf16. But a packed weight is not a usable weight: before the add/subtract/skip can run, every 2-bit code must be UNPACKED back into {-1,0,+1}. The obvious way to decode a code is to ask what it is: if the code is +1 add, else if it is -1 subtract, else skip — a chain of comparisons and selects, one per weight. On a GPU that select-chain is poison. Branch-heavy per-element logic serialises on the arithmetic units and stalls the very memory pipeline the small weights were supposed to feed — so the kernel runs compute-bound on its own bookkeeping instead of memory-bound on the 8x-smaller weights. Measured on real edge silicon (an RTX 4050, batch-1 decode, N=K=4096, bit-correct), that first select-based kernel hits only 2.6x versus bf16, and profiling shows roughly 62% of its runtime is pure unpack — the tax. Now delete the tax the same way module 1 deleted the multiply: with arithmetic, not branches. Choose the packed codes deliberately — +1 -> 0b01, -1 -> 0b11, 0 -> 0b00 — and the decode collapses to a single expression, v = (code XOR 2) - 2: code 0 -> 0, code 1 -> +1, code 3 -> -1, no comparison, no branch, two integer ops the GPU issues almost for free and hides behind the memory load. The result is bit-for-bit identical to the branchy decode — you prove it below, max error exactly 0 — but the measured kernel jumps to 5.8x faster and 3.2x more energy-efficient than bf16, landing right at the memory-bound ceiling the branchy version could never reach. That is the whole moral of the frontier in one number: the multiply-free win is a MEMORY win, and you only collect it if the unpack gets out of the way. The idea is exactly the arithmetic-over-branches move you already trust — three states delete the multiply; a clever code deletes the unpack.

03Guided practice

Worked: A ternary weight was packed into a 2-bit code to save memory. To use it you must decode back to {-1,0,+1}. The naive decode is a select-chain: np.where(c==1, 1, np.where(c==3, -1, 0)) — for every weight, test the code and pick a value. It is correct, and on a GPU it is the bottleneck: per-element branches serialise and stall the memory pipeline (measured: ~62% of the first kernel's time).

Your turn: The codes were chosen so decoding needs NO branches. With +1 -> 0b01 (1), -1 -> 0b11 (3), 0 -> 0b00 (0), the single expression (c ^ 2) - 2 recovers the weight: 0^2-2 = 0, 1^2-2 = +1, 3^2-2 = -1. Fill the gap: W_fast = (c ^ 2) - 2. One xor, one subtract, fully vectorised — no np.where.

Independent: The grader proves the point that matters: W_fast must equal both the branchy decode AND the original weights to the last bit (max error exactly 0) — a faster kernel that changed the answer would be worthless. Then read the measured ledger from the Charlot Lab kernel project (real RTX 4050 medians, bit-correct): the select-chain kernel ran 2.58x vs bf16; the branch-free decode runs 5.83x — a 2.26x tax removed — against an 8x memory-ratio ceiling. Ask yourself why 5.8x and not 8x: some unpack and reduction overhead always remains, and at batch-1 the GPU is not fully saturated. The honest number is a real 5.8x, not a slogan.

04Feedback

PASS WHEN Console prints select_err=0 and branch_free_err=0, then the measured line 'select-chain 2.58x -> branch-free 5.83x (ceiling 8x)' and 'unpack tax removed: 2.26x', ending in PASS.

If your run fails, check:

branch_free_err=1 with FAIL: the gap is unfilled — W_fast is still the placeholder zeros, so it differs from W by 1 everywhere a weight was +/-1. Replace it with W_fast = (c ^ 2) - 2.
branch_free_err is large (not 0 or 1): you wrote the arithmetic but on the wrong array or with the wrong constant. Decode the unpacked CODES c (values 0/1/3), and the expression is exactly (c ^ 2) - 2 — a bitwise xor with 2, then minus 2. Check ^ is xor, not ** or +.
You used np.where or a comparison and it passed anyway: it works, but it is the very branch this lesson deletes. The point is that (c ^ 2) - 2 needs NO select — rewrite it branch-free so the decode is two integer ops the GPU hides behind the memory load.

05Retrieve & space

The 2-bit weights are 8x smaller than bf16, yet the first kernel ran only 2.6x faster, not 8x. Name what consumed the missing ~5x, and why it is a compute problem masquerading in a memory-bound kernel.
The branch-free decode (c^2)-2 produces a result bit-identical to the branchy np.where version. Why does 'same answer, different code' translate into a 2.3x kernel speedup on a GPU but essentially nothing on a scalar CPU loop?
The codes were assigned +1->0b01, -1->0b11, 0->0b00 on purpose. Show that (c^2)-2 decodes all three, and explain why a careless code assignment (say -1->0b10) would force you back to a branch.

06Mastery & project

Passing proves you can close the loop the course has been building toward: the multiply-free win is bounded by memory movement, and realising it on real hardware means the UNPACK must be as cheap as the arithmetic. You replaced a branchy select-chain decode with a bit-exact branch-free expression, verified it changed no bits, and read the measured payoff — a ternary decode kernel that runs 5.8x faster and 3.2x more energy-efficient than bf16 on a real edge GPU, up from a stalled 2.6x.

This is the systems proof that anchors module 3: the honest joules named data movement as the wall, and here you saw the wall on a real GPU and got over it. The measured 5.8x/3.2x decode win is from the Institute's ternary kernel research program (RTX 4050, bit-correct, working-set-swept 3.9-5.7x); its portability to true edge silicon (Jetson-class, smaller cache) is the open thread. Next the module counts the silicon gates the deleted multiplier saves, then the capstone ships the end-to-end trit — the same arithmetic-over-branches instinct, one layer up.

GO FURTHER · THE LIVE PLAYGROUND

Read the whitepaper ↗ — Multiply-Free Neural Computation (TR-2026-02)
The research topic ↗ — Ternary Physical AI at the Charlot Lab

← The honest joules Silicon anyone can make →

PYTHON · NUMPY

Why the multiply-free win is a memory win

Pack ternary weights into 2-bit codes, unpack them two ways, and prove the fast way is free. The branchy select-chain decode is given; your one gap is the branch-free arithmetic decode. The grader checks it is BIT-IDENTICAL to the branchy decode and to the original weights (max error 0), then reports the real measured RTX 4050 speedups (2.58x select-chain -> 5.83x branch-free). numpy only, deterministic, < 2 s.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you