PYTHON · NUMPY

The graph is a policy

Watch a learned think block discover a fix your hand rule could not -- the step from maker to physical AI.

01Challenge

Same loop, one hidden catch: the motor is wired backwards, so your commanded +v actually moves -v. The natural hand rule -- a positive gain that steers toward the error -- now drives the wrong way and never arrives. Instead of guessing, LEARN the think block: fill in a reward that prefers reaching closer and faster, and a search over gains finds the negative gain that quietly undoes the wiring. The graph never changed shape; one block's function is now learned.

Predict firstThe motor is wired backwards. Your hand rule uses a positive gain to steer toward the target. Does it reach?

02Model

This is the whole bridge in one lab. A policy is a function from what you sense to what you do. A hand-written policy encodes what you assumed; a learned one encodes what the reward found -- including a hidden inversion you never diagnosed. That is exactly the sim-to-real gap physical AI crosses.

03Guided practice

Step A (worked): rollout inverts the motor; gain 0.5 drives away and the hand rule fails -- read it. Step B: reward() returns 0, so every gain ties and the search learns nothing -- that is the FAIL. Step C (the fix): make reward return -(abs(error)) - 0.001*steps, so closer-and-faster scores higher. Step D: the search now finds a strong negative gain that reaches -- learning discovered the inverted wiring from reward alone, with no one telling it the motor was backwards.

04Feedback

PASS WHEN PASS when your reward makes the search learn a negative gain that reaches the target.

If your run fails, check:

Read the bench’s printed result — it passes when your reward makes the search learn a negative gain that reaches the target. Run the cells top to bottom; if one errors, the FAIL line names the step to fix.
reward() still ties every gain -- score closer and faster so the search can climb.

05Retrieve & space

Define a policy in one sentence.
Why did the hand rule fail where the learned block succeeded?
What real-world gap does the hidden motor inversion stand in for?

06Mastery & project

Pass the lab: a learned think block finds the negative gain and reaches where your hand rule could not. You crossed from tuning a policy to learning one.

The centrepiece of the whole Connection Systems idea: the block graph that becomes a policy. The interactive lets you flip the quirk and train it live.

GO FURTHER · THE LIVE PLAYGROUND

Train it live: the block graph that becomes a policy ↗ — flip the hidden quirk, switch the think block to Learned, and train.

← What the wire carries The magnet, switched →

PYTHON · NUMPY

The graph is a policy

Watch a learned think block discover a fix your hand rule could not -- the step from maker to physical AI.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you