What a decision costs
Answering a decision without writing a sentence is a real energy win: at the routing accuracy the open models actually publish, the same decision costs about a fifth of what generating it costs. How much further you can go is decided in one line of algebra, and not by the model, because a decision sent to the typed engine and answered wrongly pays the typed pass and the generative fallback. Set the router's accuracy and read what a hundredfold would take.
The decision, and where you send it
One decision, two paths. E₃ = tokens × J/token is the generative baseline; E₂ is one typed forward pass with no generation. Route with accuracy p and the composed cost is E₂ + E_read + (1 − p)·E₃.
Reference points, all published or measured. 0.39 J/token LLaMA3-70B on H100 at FP8; 1.80 a mid-range estimate at 5×10−4 Wh/token; 3.50 LLaMA-65B on V100. Router accuracy 79.90% is Kev-0.5B's reported figure, on held-out splits of its own six training datasets, with out-of-source generalisation stated by its authors as not measured. Readout is on its own axis because the Institute's metered silicon puts a read at 64× the operation it reads.
What a published claim requires of your router
The ceiling depends on nothing but p. Reading it backwards turns a cost claim into a statement about a quantity the claim does not mention.
| Claim | Saving | Router accuracy it requires | Status |
|---|
A router is not free of the thing it is routing around: the miss rate is what turns a hundredfold claim into a fivefold one. This review did not locate a published out-of-distribution router accuracy for any deployed typed-decision system, which is the measurement this page is asking for.
Institute for Physical AI @ John Bailey Institute · The Charlot Lab · companion instrument to TR-2026-48. Energy figures are inputs, not simulations: every default is a published or metered number named above, and the arithmetic between them is the whole model.