PYTHON · NUMPY

Three states delete the multiply

Implement a matrix-vector product with ternary weights using only add, subtract, and skip — no multiplies — and prove it is bit-for-bit identical to x@W (max error < 1e-9).

01Challenge

Every neural network you have ever run is, underneath, a wall of multipliers — and the multiply is the single most expensive arithmetic primitive silicon builds. Here is the frontier claim, stated without a hedge: if the weights live in {-1, 0, +1}, the multiplier array is not optimized — it is DELETED. Multiply-by-a-weight collapses into "add this input", "subtract this input", or "do nothing", and the answer is not an approximation — it is exactly W@x. In this lesson you make that array disappear with your own hands.

Predict firstCan you compute Wx with ZERO multiplies and get an answer that is bit-for-bit the same as the full floating-point x@W?

02Model

Look at what a weight actually does in a dot product: it scales an input. Now constrain that weight to three values. A weight of +1 means "pass this input through untouched" — that is an ADD. A weight of -1 means "pass it through negated" — a SUBTRACT. A weight of 0 means "this connection does not exist" — nothing happens, and nothing is the cheapest operation in the universe (Horowitz, ISSCC 2014, prices even an int8 add at ~0.03 pJ; the skipped op costs zero). No coefficient is ever multiplied by a datum, because every coefficient IS one of add / subtract / skip. This is not lossy compression of a real-valued matrix — a ternary matrix is exactly itself, so the multiply-free result equals x@W to the last bit of floating point. And balanced ternary hands you a second gift the binary world cannot: negation is a sign flip with 0 mapping to 0, so flipping a weight's sign flips a single symbol instead of rippling carries through a two's-complement word — fewer bit transitions, and dynamic energy scales with transitions. The multiplier array — the densest, hottest, most fab-hungry block on an AI die — simply is not on the floorplan anymore. That is the whole game of this course, and it starts with three states and a dot product.

03Guided practice

Worked: Take one output neuron with weights w = [+1, 0, -1] acting on inputs x = [a, b, c]. The textbook dot product is 1*a + 0*b + (-1)*c. Rewrite it as selection instead of arithmetic-on-data: keep a (because w=+1), drop b (because w=0), and keep -c (because w=-1). Result: a - c. Notice there is no * touching a, b, or c — only choosing which inputs to add and which to subtract. The 0 weight didn't cost a multiply-by-zero; it cost nothing, because we never looked at b at all.

Your turn: Generalize to a whole matrix W of shape (n_in, n_out). For a fixed input vector x, broadcast it down every column with x[:, None]. Build a "positive contributions" array with np.where(W == 1, x[:, None], 0.0) and a "negative contributions" array with np.where(W == -1, x[:, None], 0.0). Convince yourself that summing the first over axis 0, minus the sum of the second over axis 0, reproduces each column's dot product — with the W == 0 entries contributing a hard zero.

Independent: In the bench, write the single out = ... line using two np.where selections and .sum(0) — no * operator anywhere on x or W. Then read off how many of the weights were zeros: that count is free sparsity, operations you never had to perform.

04Feedback

PASS WHEN The Grade cell prints PASS with max|out - x@W| on the order of 1e-15, and reports the live-computed zero-weight count (~46-47%) as free skips.

If your run fails, check:

FAIL with a large error and out is a scalar (or the shape is wrong / broadcasts oddly): you likely summed over the wrong axis. Each column is one output neuron, so contributions run down the INPUT axis — use .sum(0) (axis 0, the n_in rows), not .sum(1) or a bare .sum().
FAIL with max error near the magnitude of the surviving contributions (roughly 1-3): you almost certainly dropped the subtraction and wrote only the W==1 term, or added the two np.where blocks instead of subtracting the second. The w=-1 weights mean SUBTRACT, so it must be (+1 selection).sum(0) minus (-1 selection).sum(0).
FAIL because you multiplied — e.g. out = (W * xb).sum(0): that is mathematically correct but it is the very multiply this lesson deletes. Rebuild out from np.where selection so zero multiplies touch the data; the multiplier array must not appear.

05Retrieve & space

A weight of 0 and a weight of +1 both avoid a 'real' multiply, but only one of them is 'free sparsity.' Explain the operational difference between what the hardware does for a 0 weight versus a +1 weight.
The ternary result equals x@W to ~1e-15, not exactly 0 error. Why is there any residual at all, and why is it NOT evidence that the multiply-free method is approximate?
Suppose you replaced the ternary weights with values in {-2, 0, +2}. Could you still avoid true multiplies entirely? Describe what primitive each weight would map to and whether the multiplier array truly disappears.

06Mastery & project

Passing proves you can express a matrix-vector product entirely as data-independent selection (add / subtract / skip) and demonstrate it is numerically identical to the multiplying version — the concrete mechanism by which ternary weights remove the multiplier array without changing the answer.

This is the first stone of the course's spine: the multiply is the enemy, and three states delete it. Every later module builds on this exact equivalence — the information-theory lesson quantifies the cost of storing those three states (log2(3) = 1.585 bits/trit), the materials lesson shows why three voltage levels widen noise margin on mature nodes, and the systems lessons carry this add/subtract/skip kernel toward in-memory compute where the deleted multiply lets us also collapse the data-movement wall.

GO FURTHER · THE LIVE PLAYGROUND

Drive the live ternary policy ↗ — FP32 vs ternary vs log, trained live
Read the whitepaper ↗ — Multiply-Free Neural Computation (TR-2026-02)

← The multiply is the enemy Absmean: rounding a net to three states →

PYTHON · NUMPY

Three states delete the multiply

A ternary matvec, built from selection instead of multiplication. `x @ W` appears exactly once — as the trusted reference you must match to floating-point precision. Your job is the `out =` line: reproduce that reference using only np.where selection, addition, and subtraction. If a single `*` touches the data, you've missed the point.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you