PYTHON · NUMPY

How can a layer be vast and cheap at the same time?

In this loop the cheap path runs more often AND uses more of the state than the expensive one. What does that tell you about cost?

Separate rate, extent and cost for the two paths in this loop, and show that cheapness follows from neither of the other two.

01Challenge

The cheap path here runs every step and sees the whole state. The expensive one runs occasionally and considers a narrow set of options. Before you count: which of rate and extent predicts cost?

Predict firstIn this loop the cheap path runs more often AND uses more of the state than the expensive one. What does that tell you about cost?

02Model

Around eighty percent of the neurons in a human brain sit in the cerebellum, and glucose consumption per neuron there is roughly ten to twenty times lower than in cortex. The fast wide layer is numerically enormous and metabolically cheap; the slow narrow one is small and expensive. So cheapness does not follow from being fast, and it does not follow from being small. It follows from low precision, feedforward structure and sparse activity. Rate is set by the world, extent is set by the task, and cost has to be engineered on its own.

03Guided practice

Count the model-steps each path spends per decision. Then compute the joules and compare against your intuition about which one is the big one.

04Feedback

PASS WHEN You counted the operations and found the three axes moving independently.

If your run fails, check:

The cheap path is faster, sees more of the state, and costs 1440 times less. If rate or extent predicted cost, that combination could not exist.
Cost is set by model-steps per decision: one for the dot product, 24 by 60 for the search. That is the quantity to engineer, and it is why a large model running at 200 Hz is fast, wide and ruinous all at once.
The biological version is the cerebellum: about eighty percent of the brain's neurons at a tenth to a twentieth the glucose per neuron. Vast and cheap, because it is low precision and feedforward, not because it is fast.

05Retrieve & space

Name the three axes and say what sets each one.
The cheap path here is faster and wider than the expensive one. What does that prove about cost?
What single quantity actually sets the cost of a decision in this loop?

06Mastery & project

Given a proposed two-tier stack, state the rate, extent and cost of each tier separately and identify which the design has conflated.

Take a published dual-system robot policy. Extract the rate of each tier and estimate the cost of each in operations per decision.