Dendritic Computation: Learning Without Forgetting
The unit at the base of deep learning is a single weighted sum, and a real neuron is not. Build the dendritic unit from the ground up: see why one point neuron can't solve XOR but a dendritic one can, watch a standard network catastrophically forget a task while a context-gated dendritic network keeps it, and recognize the same dendritic gating hiding inside attention and mixture-of-experts. It ends on the question (if the richer unit is better, why did the point neuron win?) and its answer: the hardware lottery. Grounded in Beniaguev 2021, Poirazi & Mel, Numenta's Active Dendrites, Chavlis & Poirazi 2025, and Hooker 2020.
▶ Start the course ← All coursesWhere this sits, and what moves it.
Binding constraint · What one unit can compute before you pay for another. A point neuron computes one weighted sum; a dendritic tree computes a composition of them, and the difference shows up as parameters, energy, and whether the thing forgets.
Catastrophic forgetting was treated as a training problem to be managed with rehearsal buffers and regularisers. That a structural change to the unit -- context-gated dendritic branches -- addresses it directly stayed a neuroscience observation, because the arithmetic it needs is conditional and sparse and the silicon of the day charged more for that than for the dense multiply it replaces.
The demonstration is reproducible and sits in this course: the shared network forgets, the gated one does not. the part is why the better primitive still loses, and it is not that the idea is wrong. It is the hardware lottery -- a dense multiply-accumulate array is what silicon has been optimised for across four decades, and a sparse conditional tree is not.
It turns the day a substrate makes conditional, sparse routing as cheap as dense multiplication -- which is the same substrate question ternary logic and thermodynamic sampling are asking from their own directions. Watch the energy per conditional branch, not the accuracy tables; accuracy has not been the obstacle for some time.
Every hard thing was impossible until the constraint that made it impossible was named. How we read a frontier →
The unit
The point neuron won on hardware and universality; the dendrite wins on what one unit can compute.
- L2The point neuron, and why it wonOne point neuron, a weighted sum and a threshold, is asked to solve XOR. What happens?Meet the unit at the base of every network, one weighted sum and a threshold, and see both its power and its limit.→
- L3The neuron is a networkA single biological neuron, with its branching dendrites, computes about as much as…See the load-bearing fact: a real neuron's dendrites each apply their own nonlinearity, so one neuron computes what a point neuron needs a whole network for.→
Forgetting
Why a shared network forgets, and how context-gated dendrites route each task to its own sub-network.
- L3Catastrophic forgettingTrain a standard neural net on task A, then train it on task B. What happens to task A?Watch the failure that keeps today's models frozen: a network learns a second task by overwriting the first.→
- L4Dendrites that don't forgetYou let each task's context fire its own sparse set of dendrites instead of sharing all the weights. What does that fix?Fix the forgetting: let each task's context fire its own sparse set of dendrites, so tasks stop sharing weights.→
Why the better primitive still loses
The dendritic operation is already in the frontier, and the hardware lottery explains what's holding it back.
- L4The dendrite, hiding in the frontierAttention, mixture-of-experts and gated layers all turn out to share one operation. What is it?See that attention, mixture-of-experts and gated layers are the dendritic operation (multiplicative, conditional compute) in GPU-friendly form.→
- L4The hardware lotteryDendritic units compute more per neuron than the plain point neuron. So why did the SIMPLE point neuron become the basis of today’s deep networks?Answer the real question: if the richer unit is better, why did the point neuron win? Because the hardware of this era rewards dense matrix multiplies.→