PAI-135 · Education

Dendritic Computation: Learning Without Forgetting

The unit at the base of deep learning is a single weighted sum — and a real neuron is not. Build the dendritic unit from the ground up: see why one point neuron can't solve XOR but a dendritic one can, watch a standard network catastrophically forget a task while a context-gated dendritic network keeps it, and recognize the same dendritic gating hiding inside attention and mixture-of-experts. It ends on the honest question — if the richer unit is better, why did the point neuron win? — and its answer: the hardware lottery. Grounded in Beniaguev 2021, Poirazi & Mel, Numenta's Active Dendrites, Chavlis & Poirazi 2025, and Hooker 2020.

Skilled·3 modules·6 lessons
▶ Start the course ← All courses
Module 1

The unit

The point neuron won on hardware and universality; the dendrite wins on what one unit can compute.

  1. L2The point neuron, and why it wonMeet the unit at the base of every network — one weighted sum and a threshold — and see both its power and its limit.→
  2. L3The neuron is a networkSee the load-bearing fact: a real neuron's dendrites each apply their own nonlinearity, so one neuron computes what a point neuron needs a whole network for.→
Module 2

Forgetting

Why a shared network forgets, and how context-gated dendrites route each task to its own sub-network.

  1. L3Catastrophic forgettingWatch the failure that keeps today's models frozen: a network learns a second task by overwriting the first.→
  2. L4Dendrites that don't forgetFix the forgetting: let each task's context fire its own sparse set of dendrites, so tasks stop sharing weights.→
Module 3

Why the better primitive still loses

The dendritic operation is already in the frontier — and the hardware lottery explains what's holding it back.

  1. L4The dendrite, hiding in the frontierSee that attention, mixture-of-experts and gated layers are the dendritic operation — multiplicative, conditional compute — in GPU-friendly form.→
  2. L4The hardware lotteryAnswer the real question: if the richer unit is better, why did the point neuron win? Because the hardware of this era rewards dense matrix multiplies.→