PYTHON · NUMPY · IN-BROWSER

When words have order

Confront the compositional-order problem — 'red on green' vs 'green on red' — and apply the honest fix our stacking VLA uses.

01Challenge

Show that order-blind embeddings can't tell the two orderings apart, then parse the source block instead.

02Model

Manipulation instructions carry order: 'stack the red on the green' is not 'stack the green on the red'. But mean-pooled sentence embeddings barely encode order — we measured it at 0.67, near chance for that minimal pair — so a policy conditioned only on the embedding would put the wrong block down about a third of the time. The honest move is not to pretend otherwise but to parse the order: the first colour named is the source, and the policy is conditioned on the resolved source and target. That is exactly what our nano-vla-stack does, and its 94 percent stack success is why the choice was right.

03Guided practice

1. See the blindness. A bag-of-words embedding gives 'red on green' and 'green on red' the same vector. 2. Parse instead. The first colour word is the source; the other is the target. 3. Condition on that. The policy is told the resolved source and target, so the manipulation is the real work — not a language guess it can't make. Honest engineering beats a false claim.

04Feedback

PASS WHEN PASS when you confirm the embeddings are identical and correctly parse the source colour.

A cell errored — run them top to bottom.
The two orderings embed identically; parse the first colour as the source.

05Retrieve & space

Why can't a mean-pooled embedding handle 'red on green' vs 'green on red'? (it barely encodes word order).
What is the honest fix our stacking VLA uses? (parse the first colour as the source and condition on it).
What did we measure the order-classification at? (~0.67, near chance).

06Mastery & project

You can recognize where an embedding fails and engineer honestly around it.

A research reflex: when a component can't do a job, measure it, say so, and design around it — don't ship the false claim.

← Does it obey?Motion needs memory →

identical:\",bool(np.allclose(e1,e2)))","label":"Order-blind embeddings"},{"code":"# the honest fix (what nano-vla-stack does): PARSE the order — first colour = source.\ndef source(s):\n for w in s.split():\n if w in ('red','green'): return w\nprint('parsed source of \"stack the green on the red\":', source('stack the green on the red'))\nassert np.allclose(e1,e2) and source('stack the green on the red')=='green'\nprint(\"PASS - order-blind embeddings can't separate the two, so we parse the source and condition the policy on it. Honest engineering beats a false claim.\")","label":"Parse the source — the honest fix"}],"intro":"Expose order-blindness, then parse the source to fix it.","key":"vision-language-action/order","kind":"python","title":"When words have order"}">

PYTHON · NUMPY · IN-BROWSER

When words have order

Expose order-blindness, then parse the source to fix it.

You read

the arrays and values already in scope

You change

the code you write in each cell

Fixed

the dataset and the checks that grade you