PYTHON · NUMPY

Bits, trits, and the best radix

Compute that one trit carries log2(3)=1.585 bits and that base 3 beats base 2 on radix-economy cost averaged over a large range of integers.

01Challenge

Every machine you have ever used bets everything on two states. But information theory never agreed that two was the answer — it only ever cared about symbols and how much each one carries. In this lesson you will compute the number the whole field keeps circling back to: log2(3). When BitNet announced "1.58-bit LLMs," that was not marketing rounding — it was the trit, rediscovered by the AI industry the same way Setun found it in 1958. Prove the trit is the information-optimal atom, and the rest of this course is just building the machine that deserves it.

Predict firstWhich integer base is the most economical for representing numbers (radix economy = radix x number-of-digits)?

02Model

Start from Shannon, not from silicon. A symbol drawn from an alphabet of size r carries log2(r) bits — that is the definition, not a convention. A bit (r=2) carries exactly 1. A trit (r=3) carries log2(3) = 1.585 bits: 58% more information per symbol, for free, before any circuit exists. Now ask the deeper question the field of radix economy asks: if it costs you (radix x number-of-digits) to build a representation — more digit-positions cost hardware, and a larger radix costs more distinguishable levels per position — which base is cheapest? Minimize r/ln(r) and the answer is e = 2.718, the natural logarithm's own base. You cannot build 2.718 states, so you round — and 2.718 rounds to 3, not 2. Three is the most economical INTEGER radix; binary only wins for tiny ranges where a single extra digit is decisive. This is the quietly radical fact of the course: the information-optimal atom of computation is the trit, and industry keeps voting for it without admitting it. GDDR7 memory ships PAM-3 with three levels {-1,0,+1} because three levels buy ~50% more voltage margin than four. USB4 v2 packs 11 bits into 7 trits (3^7 = 2187 >= 2048). BitNet's weights live in {-1,0,+1} and its authors call it '1.58 bits' — which is literally log2(3). Setun computed in balanced ternary in 1958; Shannon wrote a symmetrical notation for exactly these three symbols. The trit was never the fringe. Binary was the historical accident that a two-state transistor made cheap to manufacture — and this course is about what happens when you stop paying the binary tax and build for the atom information theory actually prefers.

03Guided practice

Worked: Bits per symbol in radix r is np.log2(r). For binary, np.log2(2) = 1.0 — one bit, exactly as named. Watch the pattern across bases: np.log2([2,3,4,10]) gives [1.0, 1.585, 2.0, 3.322]. Notice the trit already beats the bit by 0.585 of a bit per symbol.

Your turn: Fill the one gap — set bits_per_trit = np.log2(3). This is the number the whole course orbits. Confirm it prints ~1.585, and confirm it equals the entry for radix 3 in bits_per_symbol.

Independent: The radix-economy cost is E(r,N) = r * ceil(log_r(N)), where log_r(N) = np.log(N)/np.log(r). Run it for every N from 2 to 1000 for base 2 and base 3, take the means, and check that base 3's mean cost is <= base 2's. You are proving, numerically, that three is the cheaper integer base once the range is large — the claim e~2.718 rounds to 3 predicts.

04Feedback

PASS WHEN Console prints bits_per_trit = 1.5850 and PASS, with base3 mean cost (~17.74) <= base2 mean cost (~17.97).

If your run fails, check:

If bits_per_trit prints 0.0000 and you see 'FAIL: set bits_per_trit = np.log2(3)', the gap is still at its placeholder 0.0 — you left the starter value. Replace it with np.log2(3); that evaluates to 1.585, the base-3 entry of bits_per_symbol.
If bits_per_trit prints 1.0986, you wrote np.log(3) (natural log) instead of np.log2(3) (base-2 log). Radix economy uses natural log for the change of base, but bits-per-symbol is always measured in base-2 — bits, by definition. Switch to np.log2.
If you see 'FAIL: base 3 should beat base 2 on average', the radix_economy helper was edited: it must keep np.ceil (whole digit positions) and np.log(N)/np.log(r) for log_r(N). Dropping the ceil, or dividing by np.log2(r) while the N-log stays natural, corrupts the digit count so the two means no longer compare correctly — restore the helper as given.

05Retrieve & space

BitNet describes its {-1,0,+1} weights as '1.58 bits.' Where does 1.58 come from, and what would the exact value be?
Radix economy is minimized at base e~2.718, yet we build machines in base 2. Explain why 3 is the best INTEGER radix and why binary still wins for very small ranges.
GDDR7 chose PAM-3 (three voltage levels) over PAM-4 (four). State the information-theoretic and material trade-off in one sentence: what does the third-fewer level buy you?

06Mastery & project

Passing proves you can derive, not just recall, the course's founding number: a trit carries log2(3)=1.585 bits, and base 3 is the most economical integer radix over a large range. You have shown from information theory alone — before any transistor — that binary was a manufacturing accident, not an information-theoretic optimum.

This is the information-theory floor the whole course stands on. Module 1 establishes WHY the trit is the optimal atom; later modules show HOW three states {-1,0,+1} delete the multiply (math), why three levels self-regulate and hold noise margin on mature nodes (material science), and how that collapses the energy-per-inference bill for embodied AI at the edge. Every 'ternary wins' claim downstream traces back to the 1.585 you computed here.

GO FURTHER · THE LIVE PLAYGROUND

Drive the live ternary policy ↗ — FP32 vs ternary vs log, trained live
Read the whitepaper ↗ — Multiply-Free Neural Computation (TR-2026-02)
See three-level quantum info ↗ — a qutrit carries log2(3) bits in the quantum world too

← Frontiers in Physical AI: Ternary The multiply is the enemy →

PYTHON · NUMPY

Bits, trits, and the best radix

Numpy only, deterministic, under two seconds. Two computations: (1) how many bits a symbol carries in each radix, and (2) the radix-economy cost of representing every integer up to N. Fill the single gap, then run — PASS confirms log2(3)~1.585 AND that base 3 beats base 2 on average.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you