FERROMOTION · RUST → WASM

The universal approximator

A single neuron can only draw a straight boundary. What does stacking them into layers buy you?

Stack neurons into a network and watch it bend onto any continuous curve. Meet the multilayer perceptron and the theorem that makes it powerful, and the catch that makes physics necessary.

01Challenge

A single neuron computes a weighted sum passed through a nonlinearity, it can only draw a straight boundary. Stack them into layers and something remarkable happens: the universal approximation theorem says a network with one hidden layer of enough tanh units can approximate ANY continuous function to any precision. The Bench trains a real ferromotion Mlp (through the same autodiff tape from Lesson 1, with the Adam optimizer) to fit a target curve. Press Train and watch the green network curve bend onto the gold target as the loss falls. Then switch the target to the kink and the step and fit those too. This is the black box: it can fit anything you show it, which is exactly why, with little data, it also learns nonsense between the points.

Predict firstA single neuron can only draw a straight boundary. What does stacking them into layers buy you?

02Model

A perceptron is a weighted sum plus a bias, squashed by a nonlinearity like tanh; alone it draws one hyperplane. Stack perceptrons into layers and the compositions of tanh bumps can be summed to trace any continuous shape, Cybenko's universal approximation theorem. Depth helps: each layer composes features from the last, so expressivity grows fast with layers. Training is exactly Lesson 1's loop, measure the mean-squared error, get the exact gradient from the tape, step every weight downhill with Adam (which adapts a per-weight step size). But note what the network does NOT have: any notion of energy, momentum, or the equations of motion. It fits the training points and interpolates smoothly, yet outside the data it can do anything at all. That blind extrapolation, plus the appetite for data, is the whole reason the rest of this course injects physics.

03Guided practice

Step A (fit the wave): press Train ×1000. The green curve snaps onto sin(3x) and the MSE drops toward 1e-3. Step B (fit a kink): switch the target to |x|, a function with a sharp corner, and train again; the smooth network rounds the corner slightly but still fits. Step C (fit a step): switch to the step; the network struggles most here (a discontinuity is hard for a smooth net) yet still approximates it. Step D (the lesson): the same ~480-parameter network fit three very different shapes with no change to its structure, it assumes nothing. Powerful, and dangerous: it would fit noise just as happily.

04Feedback

PASS WHEN PASS when the network fits the target curve, proof that a plain MLP can approximate any continuous function.

If your run fails, check:

Press Train ×1000 and watch the green curve bend onto the gold target.
If it plateaus above the threshold, train again, deeper fits take more epochs, and the step/kink targets are harder than the wave.

05Retrieve & space

What does the universal approximation theorem promise, and what does it NOT promise about data outside the training points?
Why can a single perceptron not fit XOR, while a network with a hidden layer can?
The network fit the wave, the kink, and the step with no structural change, why is that flexibility also a liability?

06Mastery & project

You trained a real neural network to approximate any curve you gave it, the black-box baseline. It assumes nothing about the physical world, which is its power and its flaw. Every remaining module hands it a physics prior so it needs less data and extrapolates safely.

Sets up the central question of the course: where do we inject physics, into the data, the loss, or the architecture? The next module answers with the priors themselves; the modules after put each one to work.

GO FURTHER · THE LIVE PLAYGROUND

The origin of this idea: TensorFlow Playground ↗ · the classic in-browser MLP the physics-informed ML course used to show a domain feature beating brute force.

← The machine that follows gradients The equation of motion →

FERROMOTION · RUST → WASM

The universal approximator

Live ferromotion-wasm: train a real MLP (autodiff tape + Adam) to fit a curve. Drive the MSE down and watch the network approximate the target.

You read

the readout: what the Rust crate computes, recomputed as you change things

You change

the bench controls, and with them the assumption the measurement rests on

Fixed

the ferromotion crate doing the computing, and the check that decides pass