PYTHON · NUMPY

Learn the flow, predict the future

Learn a system's continuous dynamics by training THROUGH an ODE solver — the Neural ODE — and watch its rollout predict the trajectory past the data it was trained on. Meet the integrator-in-the-loop idea behind Variational Integrator Networks.

01Challenge

Every method so far predicted a value. A Neural ODE predicts a FLOW. Instead of learning 'next state from this state', it learns the derivative — the vector field ẋ = f_θ(x) — and gets trajectories by handing f_θ to an ODE solver, exactly like ferromotion integrates a robot. Training is the striking part: you integrate the learned field forward, compare the whole rollout to the observations, and backpropagate through the solver itself — every RK4 step is on the autodiff tape. The Bench trains a Neural ODE on the FIRST part of one damped-oscillator trajectory (the solid gold curve), then rolls its learned field forward across the entire horizon. Watch the green rollout lock onto the trajectory and keep tracking it into the dashed future — the part it never saw. It can extrapolate because it learned the flow (the grey arrows), not a table of points.

02Model

A residual network learns x_{t+1} = x_t + f_θ(x_t): a fixed-step update. A Neural ODE takes the limit — it learns the continuous derivative ẋ = f_θ(x) and integrates it with a real ODE solver, so you can query any time, use adaptive steps, and treat depth as continuous. Training matches an integrated trajectory to data; the gradient of that loss w.r.t. the field's parameters flows back through every step of the integrator. Two ways to get it: the continuous adjoint (solve a second ODE backward in time — memory-cheap) or, as here, the discrete adjoint (record the whole rollout on the reverse-mode tape and backpropagate — exact and simple). The reason a Neural ODE extrapolates where a point-predictor cannot: it learned the vector field over a region, so integrating forward past the last data point still follows the true flow. And because the integrator is PART of the model, its choice matters — this is the idea behind Variational Integrator Networks: put a structure-preserving (symplectic) integrator in the loop and a learned conservative system will hold energy on long rollouts, exactly the drift lesson from Module 2, now baked into the learner.

03Guided practice

Step A (before): the untrained field's rollout (green) shoots off — it knows nothing. Step B (train through the solver): press Train ×1500. The green rollout snaps onto the gold trajectory. Step C (see the extrapolation): the solid gold is the training window; the dashed gold is the future the model never saw. The green rollout keeps following it — the extrapolation error drops below a few hundredths. Step D (read the flow): the grey arrows are the learned vector field. The trajectory rides the flow; because the model learned the flow everywhere nearby, it can predict where the trajectory goes next.

04Feedback

PASS WHEN PASS when the learned field's rollout tracks the trajectory beyond the training window — it learned the flow, so it can predict the future.

If your run fails, check:

Press Train ×1500 and watch the green rollout lock onto the gold trajectory and follow it into the dashed future.
The grey arrows are the learned vector field — the flow the trajectory rides on.

05Retrieve & space

What does a Neural ODE learn instead of a next-state map, and why does that let it predict past the training window?
What does 'backpropagate through the solver' mean, and what is the discrete adjoint used here?
Why does the choice of integrator become part of the model — and how does that lead to Variational Integrator Networks?

06Mastery & project

You trained a model of continuous dynamics by differentiating through an ODE solver, and watched it predict a trajectory it was never shown — because it learned the vector field, not the points. The integrator lives inside the model, which is the seed of structure-preserving Variational Integrator Networks.

This closes the architecture module and the arc from Module 2: energy drift, then conservation by construction (HNN), then structure recovered from data (DeLaN), then the integrator itself inside the learner (Neural ODE / VIN). Module 5 turns to discovering equations directly from data; Module 6 puts every learned model to work in differentiable control.

GO FURTHER · THE LIVE PLAYGROUND

Neural Ordinary Differential Equations (Chen et al., 2018) ↗ — the paper that introduced Neural ODEs and the adjoint method for backpropagating through a solver.

← Learn the structure, not the map Discover the equation →

PYTHON · NUMPY

Learn the flow, predict the future

Live ferromotion-wasm: a Neural ODE learning a vector field by backprop through an RK4 solver. Train on part of a trajectory and watch its rollout predict the rest.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you