PYTHON · NUMPY · IN-BROWSER

Act: build the VLA

Combine vision and language into a policy that sees both targets, obeys the instruction, and acts toward the named one.

01Challenge

Assemble a vision-language-action policy and confirm its action points at the target you named.

02Model

Now the loop closes. You have vision that finds targets and language that names one; the action half joins them into a single function: from a frame and an instruction, out comes a move. This is the definition of a VLA, and you are about to build one in a dozen lines. It is deliberately transparent — the vision is a colour locator, the action a step toward the chosen target — so you can see every part. Our released nano-VLAs learn this same mapping from data with a convolution stack and an MLP; the shape of the thing is exactly what you build here.

03Guided practice

1. The scene. Two targets, red and green, plus the end-effector. 2. Locate both. The vision half finds each colour's position. 3. See → be told → act. Pick the target the instruction names, and step toward it. That composition — vision and language into one action — is a vision-language-action policy.

04Feedback

PASS WHEN PASS when the action aligns with the named target (cosine > 0.9).

A cell errored — run them top to bottom.
The action should point at the target you named.

05Retrieve & space

What are the three parts of a VLA? (vision, language, action).
What does the action half output here? (a step toward the chosen target).
How do our released nano-VLAs learn this mapping? (from data, with a conv stack + MLP).

06Mastery & project

You built a working vision-language-action policy — see, be told, act.

The whole idea in one function: a frame and an instruction go in, a move comes out.

GO FURTHER · THE LIVE PLAYGROUND

Run the released nano-VLA ↗ — Type an instruction and watch the real released policy drive the arm — the trained version of what you just built.

← Be told: ground the instruction Does it obey? →

0.9, \"the action should point at the named target\"\nprint(\"PASS - see, be told, act. You just built a vision-language-action policy.\")","label":"The VLA: see → be told → act"}],"intro":"Compose vision + language into an action and verify it aims at the named target.","key":"vision-language-action/action","kind":"python","title":"Act: build the VLA"}">

PYTHON · NUMPY · IN-BROWSER

Act: build the VLA

Compose vision + language into an action and verify it aims at the named target.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you