An open-vocab robot arm you drive with words.

Switch red and green and the gold crosshair jumps to the block you named, the policy localizing its target from pixels alone, to within a few percent of the frame, and the arm reaching the spot it marked. Every node is real and the whole forward pass runs on your device. Locating the named target, then reaching it, is one way a VLA works.

Held-out, full-episode: it grasps, carries, and stacks the correct block 94.0% of the time. Flip which block is the source and success against the original target drops to 22.8%, the language is doing the work. 55,666 parameters; the whole two-phase forward pass runs in plain JavaScript. Weights: ↗ nano-vla-stack.

This panel is a replay on captured held-out frames, which is how the policy was evaluated. The live version, the same weights closing the loop on the sim's own render, is directly below. Weights + the capture recipe: ↗ nano-vla-pixels.

We had written this off as impossible, and half of that was right: a page cannot read its own WebGPU canvas, during the render loop the 2D path comes back blank, which we re-confirmed. The way through is a render target rather than the canvas: draw the same scene into one and readRenderTargetPixelsAsync returns true pixels. Three corrections put those pixels in the policy's distribution, de-pad the rows (WebGPU aligns them to 256 bytes), encode to sRGB (the target renders linear), and match the canvas's antialiasing. Against a real capture of this canvas that lands at MSE 0.0037; flipping the rows or skipping the sRGB step lands 10 to 16 times worse.

Once the readback worked we could do something the old capture pipeline could not: train the policy on the very frames it would run on, collected through this readback path with the expert's action perturbed so the arm visited states a clean demonstration never contains. Train and live stopped being two distributions. Driving from pixels alone, this policy now completes about 23 reaches a minute, the state-based expert it was cloned from does 21. It matches its teacher, seeing only the picture the teacher never needed.

That last number cost us an honest lesson, so here it is. For most of this work the same policy scored about one reach a minute and read like a hopeful failure: a student, not the expert. The cause was not the policy. The loop re-perceived every 90 milliseconds, a constant typed early and never questioned, so the policy decided at roughly 11 Hz while the expert re-decided every frame at 60; its action went stale between glances and the arm overshot. Setting the interval to 30 milliseconds (still 9 ms of real work, nowhere near saturated) took it from ~1 to ~23 reaches a minute. One arbitrary number had been the ceiling the whole time, and it had silently poisoned every experiment run through it: models we had ranked as broken and as best were, measured correctly, the same. What actually moved the needle was the readback capture, ~10 reaches a minute up to parity; several rounds of the fancier data-collection we had credited moved nothing. The full corrected scoreboard, including what we got wrong: ↗ nano-vla-pixels-dart.

The novel-instruction number is the open-vocab proof: unseen phrasings still work, because the on-device text encoder maps them near the ones it learned. The flipped number is the language-grounding proof: with the wrong instruction it drives to the other block, so success at reaching the named one collapses to zero. It was trained by behavior cloning on a scripted Jacobian-IK expert and made robust to closed-loop drift with DART-style state-noise injection, the same idea as DAgger. A small reference policy on a rendered task, not a foundation model.