PYTHON · NUMPY

Quantize for the Edge

Quantize a float matmul to int8 and show the accuracy holds — the move that lets Physical-AI inference run in a fraction of the power and memory.

01Challenge

Float math is expensive in silicon; an embodied device can't afford it. Quantize the weights and inputs to 8-bit integers, do the matmul in integer MACs, rescale, and show the result stays within a few percent of the float answer.

Predict firstYou replace 32-bit float weights and inputs with 8-bit integers and do the matmul in integer MACs. What happens to accuracy?

02Model

A robot or wearable runs on a battery, and a float multiplier costs far more area and energy than an integer one. So edge AI quantizes: map real values onto 8-bit integers with a single scale factor, do the whole matmul in cheap integer MACs, then rescale once at the end. You cut memory 4× versus float32 and use dramatically less energy per MAC — and for inference the accuracy loss is usually a couple of percent. This is the SWaP-C bargain (size, weight, power, cost) that makes Physical AI fit on the device, and it is why the accelerators you'd design are integer machines.

03Guided practice

Worked: quantize a tensor to int8 with a max-abs scale. Faded: run the integer matmul and rescale. Independent: measure the relative error against the float reference.

04Feedback

PASS WHEN You ran a neural layer in cheap 8-bit integers and kept the accuracy — the exact trick that fits inference onto a Physical-AI device.

If your run fails, check:

FAIL: error too high — the scale should be max-abs/127, and you rescale the integer result by ws*xs.
FAIL: overflow — accumulate int8 products in int32 before rescaling.

05Retrieve & space

Why quantize for the edge? (Integer MACs use far less area/energy than float, and int8 uses 4× less memory than float32.)
What is the single scale factor for? (Mapping real values onto the int8 range and back with one multiply.)
What does SWaP-C stand for? (Size, weight, power, cost — the edge design budget.)

06Mastery & project

Quantizes a matmul to int8, executes it in integer MACs, and bounds the accuracy loss (L4: the quantization trade behind edge-AI silicon).

An int8 MAC tile like this is exactly the kind of design the Module-5 capstone tapes out on TinyTapeout.

← The MAC Array Verify the Machine →

PYTHON · NUMPY

Quantize for the Edge

Quantize to int8, do the integer matmul, and measure the accuracy kept.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you