Vision-Language-Action.
The incumbent path of embodied AI in one idea: a single model that takes an instruction and a camera view and emits an action. These are VLA models — vision-language-action policies, where robot learning, foundation models, and control converge. This primer is how to read them: the loop, why action chunking matters, why data is the bottleneck — and the structural gap every one of them shares, no guarantee before the action commits, which is exactly what a physics-first certificate is built to fill.
A literacy module of the Institute for Physical AI.
See it, ground it, do it.
A VLA runs one loop: perceive the scene, ground the words to an object, execute an action toward it, then perceive again. Pick an instruction below and watch the three stages run — vision, language, action — end to end, on the page.
The grounding shown here is transparent attribute-matching so the weights are legible; a trained VLA learns that mapping from demonstration data. The action is a real executed chunk of Δx·Δy·grip steps.
Three stages, one model.
The three stages aren't three networks bolted together — a VLA is usually one transformer that ingests vision and language tokens and emits action tokens. Reading it as three stages is how you reason about where it can fail.
Encode the scene
Camera frames become a field of visual tokens — patches the model attends over. This is where the objects, their poses, and the free space around them enter the policy.
Ground the instruction
The instruction is tokenized and grounded to what the vision saw: which object, which surface, which relation. A capable policy learns this mapping from data; the demo shows it as transparent attribute-matching so you can watch the weights.
Emit an action chunk
The policy outputs not one step but a chunk — a short horizon of end-effector deltas and gripper states — then executes it and re-perceives. Chunking is what makes VLA control smooth and reactive instead of twitchy.
The scaling law points at robots, not text.
Language models scaled on text scraped from the web. There is no equivalent web-scale corpus of a robot's own actions, so VLA progress is gated by robot-interaction data: teleoperated demonstrations, cross-embodiment datasets, and simulation. The open question of the field is the data recipe — how much real vs. simulated, how to transfer across bodies, and how to learn from a policy's own experience rather than pure imitation. Reading a VLA result means reading its data first.
Where the frontier is.
The named systems that define VLA today, so you can place any new result against them.
· OpenVLA — an open 7B vision-language-action model, the common reference point.
· π0 / π*0.6 — flow-based action experts and policies that learn from their own experience, not only imitation.
· RT-2 — the result that first framed web-scale vision-language knowledge transferring into robot actions.
· NVIDIA Cosmos — world-action models that reason, generate, and act in one system.
Read it here. Build one, then run the real ones.
This primer is how to read a VLA. The next two steps make it yours: a hands-on course where you build the vision, the language grounding, and the action head yourself — with the honest problems we hit baked in as lessons — and a page where the Institute's released VLAs run live on your device: a tiny open-vocab arm, a multi-step manipulator, a policy trained on real sim pixels, and the actual 450M SmolVLA.
Interaction as the objective — grounded and metered.
The lab's research takes the axis these leave open. Where most VLAs optimize task-completion, the Charlot Lab's Ambit makes interaction itself the objective — a policy that, by construction, builds the repertoire it needs for situations it was never given. And every decision runs on MathGround, so each action carries a joules receipt and a replayability class. VLA you can afford, and trust.