PYTHON · NUMPY

Learn to Fly

Tune a control gain by trying and scoring rollouts, beating a fixed baseline — the seed of learned control.

01Challenge

A first-order system must settle to a target. A fixed baseline gain leaves lingering error. Search over gains by scoring simulated rollouts and find one that settles faster and tighter — without deriving it by hand.

02Model

Sometimes you can't hand-derive the best controller — the plant is messy, or the objective is what you care about, not the math. So you let performance pick: define a score (settle fast, end near the target), try candidate policies, keep the best. That's the whole idea behind reinforcement and policy search, stripped to one knob. Here you'll sweep a single gain, score each rollout, and let the data choose a policy that beats the baseline — the same loop that trains a neural policy, just smaller.

03Guided practice

Worked: simulate the plant and score a rollout. Faded: sweep candidate gains, keep the lowest cost. Independent: confirm the learned gain beats the baseline score.

04Feedback

PASS WHEN A scored search over gains finds a policy that beats the fixed baseline — control learned from performance, not derivation.

If your run fails, check:

FAIL: nothing beats baseline — the search range is too narrow, or the cost rewards the wrong thing.
FAIL: the 'best' is unstable — very high gains can score low here but blow up on a real plant; cap the range.

05Retrieve & space

What replaced hand-derivation here? (A score plus a search — the essence of policy learning.)
Why can a learned policy beat a tuned PID? (It optimizes the objective you actually care about, including effects you didn't model.)
What's the risk of pure score-chasing? (Overfitting to the sim — the sim-to-real gap.)
Where do real robot-learning datasets live? (Shared formats like the LeRobot dataset and the Open X-Embodiment corpus, so recorded trajectories train policies across many robots.)

06Mastery & project

Finds a control gain by scoring rollouts that beats a fixed baseline (L3: optimize a policy from performance rather than derivation).

The search loop is the seed of the capstone's learned coordination policy, trained in sim before it flies.

← Plan a Mission Formation Without a Leader →

PYTHON · NUMPY

Learn to Fly

Search over gains by scoring rollouts and beat the baseline.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you