PYTHON · NUMPY · IN-BROWSER

Be told: ground the instruction

Ground colour words in an embedding space so novel phrasings still land on the right target — open-vocabulary language in miniature.

01Challenge

Fit a grounding boundary over colour-word embeddings, then confirm words it never saw ground to the correct colour.

02Model

A VLA doesn't match strings; it grounds meaning. Words become vectors in an embedding space where 'crimson' sits near 'red' and far from 'green', and the policy learns a boundary over that space. The payoff is open vocabulary: type a word the policy never trained on and, because the embedding places it sensibly, it still lands on the right side. Here you use toy four-dimensional embeddings to see the mechanism; our released nano-VLAs use a real 384-dimensional encoder (all-MiniLM) running on your device. Same idea, more dimensions.

03Guided practice

1. Embeddings. Colour words become vectors; similar colours sit near each other. 2. A boundary. A linear fit separates red from green. 3. Open-vocab. Words never seen in training — 'ruby', 'forest' — still land correctly, because the embedding placed them near the words that were. That is what makes 'grab the crimson one' work when you only trained on 'red'.

04Feedback

PASS WHEN PASS when every novel colour word grounds to the correct side of the boundary.

A cell errored — run them in order.
Novel words should ground to the right colour.

05Retrieve & space

What lets a VLA handle words it never trained on? (the embedding places new words near familiar ones).
What does the grounding boundary separate? (which target the instruction names).
What real encoder do our nano-VLAs use for this, on-device? (all-MiniLM-L6-v2, 384-D).

06Mastery & project

You can ground open-vocabulary instructions to targets — the language half of a VLA.

The reflex behind 'talk to the robot': meaning lives in an embedding, so new phrasings just work.

← See: pixels to a place Act: build the VLA →

0) # 0=red,1=green\nprint(\"trained a language grounding boundary over\",len(X),\"words\")","label":"Toy embeddings + a grounding boundary"},{"code":"# open-vocab test: words the boundary NEVER saw should still land on the right side.\nNOVEL={'ruby':([.9,0,.15,.05],0),'rose':([.85,.05,.2,0],0),'forest':([.05,.7,.1,.15],1),'mint':([0,.85,.15,.1],1)}\nok=0\nfor word,(vec,truth) in NOVEL.items():\n pred=ground(vec); print(f' \"{word}\" -> {\"green\" if pred else \"red\"}'); ok+= (pred==truth)\nassert ok==len(NOVEL), \"every novel word should ground to the correct colour\"\nprint(\"PASS - novel words ground correctly. That is open-vocabulary language, in miniature.\")","label":"Open-vocab: novel words ground correctly"}],"intro":"Ground colour words and test open-vocabulary generalization.","key":"vision-language-action/language","kind":"python","title":"Be told: ground the instruction"}">

PYTHON · NUMPY · IN-BROWSER

Be told: ground the instruction

Ground colour words and test open-vocabulary generalization.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you