Hall Lab · instrument
Does the loop sample?
Every human-facing interface use case we could enumerate, scored on one question: does the thing the user is waiting for require drawing a sample from a constrained distribution? That decides whether alternative hardware is relevant at all. It is not sufficient on its own, and the section at the foot of this page says what the second question is and why we did not have it when we started.
Grant Markhart, Industrial Research Fellow · The Hall Lab, with The Charlot Lab
102 use cases, not 100: trimming two to reach a round number would be choosing the number over the survey.
samples the user waits on a draw from a constrained distribution partly constrained, but solved to one answer rather than drawn no display, control, recognition or retrieval
How this was scored, and the one weakness you should hold against it
The rule. A use case scores samples when the step the user is waiting on has many valid answers and the machine must produce one of them under constraints. It scores partly when the problem is constrained but is solved to a single answer by iteration rather than drawn. It scores no when the answer is determined by the input: display, comparison, retrieval, classification, recognition, rendering, control and sorting.
The categories are sourced; the enumeration is ours. Domain taxonomies come from industrial HMI literature in German and Japanese, automotive and XR interaction surveys, and Chinese 2026 human-computer-interaction reviews. The individual entries are a structured enumeration within those categories rather than 102 separately cited claims, which is how survey inventories are built and is worth knowing before quoting a count.
The second question, and how it was scored differently. The section at the foot of this page adds a second axis: is the per-step work a dense matrix multiply, or constraint-shaped? That one was NOT scored by one rater. The 55 rows that scored samples or partly went through two independent raters who never saw each other's calls, each told explicitly that the easier error was to answer “constraint” because it is the word the question is looking for, and each asked to judge what ships rather than what is expressible. They agreed on 50 of 55; the five that split were adjudicated one at a time and are marked • in the table. The result contradicts what this page originally claimed as its headline case, which is recorded rather than removed.
One rater, holding a thesis. Every row was scored by the same person, who expected the answer to come out interesting. That is the standard setup for confirmation bias and there is no second rater and no agreement statistic, so 39% is our reading and not a measurement of the world. The rule above is stated in full precisely so a disagreeing reader can re-score the table and say which rows we got wrong. Doing that would be the most useful possible reply to this instrument.
| Domain | Use case | The inner loop | Samples? | Per-step work | Why |
|---|
The split is by verb, not by industry. The rate barely moves across domains (from 30% in industrial process control to 57% in the home) and that narrowness is the finding. Sampling-shaped work is not a niche at the entertainment end of the market. It is roughly a third of every interface domain there is.
What separates the two columns is what the interface is being asked to do:
Draws a sample
assign · schedule · place · order · partition · complete · reconstruct · generate · resolve a conflict
Does not
display · compare · retrieve · classify · recognise · transcribe · render · control · traverse · sort
So the reason nobody uses an alternative architecture is not that there is nothing for it to do. There is a great deal for it to do, in every domain, and it is not reachable, because every interface to these machines asks the user to know what the machine is.
The unlock list
Read the second question first, at the foot of this page. All four applications below are matmul-shaped per step, so the criterion argued there says the GPU has already won them. That is not a reason to delete this list: it is a different argument for the same hardware, resting on different and weaker evidence, and the difference is set out below rather than papered over.
Scoring says where an alternative architecture could help. It does not say where anyone needs it. These four are different: each is an application people are actively trying to ship, each has its blocker stated in the 2026 literature in plain numbers, and in every one of them the blocker is the same thing.
| Application | Wanted for | The blocker, as stated | Shape |
|---|---|---|---|
| Diffusion policies for robot manipulation | A robot that generalises to a task it was not scripted for | Inference is sequential and expensive, needing many denoising steps; the latency is “a major limitation for real-time applications in robotics, where fast and reliable action generation is critical.” On mobile platforms it is blocked outright by compute and memory footprint. | step count |
| On-device generative imaging | A phone that edits, restores and composes without sending your photos away | Constrained by memory, battery and thermal limits. Heavy workloads push batteries into high-discharge regimes where efficiency drops and heat rises. The industry answer in 2026 is to send the hard half to a datacentre. | energy |
| Live generative content in shipped games | A world that answers what a player thinks of, not what a designer authored | Three reasons it stays in research: latency you can feel, a per-request cost on every session, and output that drifts out of consistency the moment two things must match. | latency, cost, consistency |
| Molecular conformer generation | Searching what a molecule can actually do, not what one pose suggests | Models iterate a thousand timesteps to sample a single conformer, and the trade-off between generation quality and speed is called a prevailing issue. | step count |
One blocker, four times. None of these is blocked on accuracy, or on whether the method works, or on anyone believing in it. Every one is blocked on the same quantity: the number of sequential steps it takes to draw one sample, multiplied by what a step costs in joules and milliseconds. A thousand steps to move one molecule. Many steps between a robot seeing and a robot acting. A per-session bill on every player.
That is the specific thing a machine which samples by relaxing rather than by iterating is for. Not “optimisation problems” in the abstract: this, with the cost written down by the people who cannot ship it.
The substitution is published, and its magnitude is not measured. Denoising thermodynamic models replace each neural denoising step with a Gibbs sample from a hardware energy-based model, so the mechanism connecting these blockers to this hardware is real rather than an analogy. The reported figure (parity with a GPU at about ten thousand times less energy) comes from simulation, on a small-scale image benchmark. Treat the mechanism as established and the number as a projection, because that is what each of them is.
And in three of the four the architecture is already invisible. Nobody using a robot, a phone camera or a game knows a sample was drawn, and none of them would have to learn. The machine is hidden by the product before it is built, and what is visible today is only the cost, which is the one thing an alternative architecture is supposed to change.
The fourth is weaker and should be marked as such. A computational chemist running conformer generation knows perfectly well that a sample is being drawn, so that application is a case of a cost being unaffordable rather than a machine being invisible. It stays on the list because the blocker is identical, and it is flagged here because a criterion that quietly admits its fourth case is a criterion nobody can use.
What ships anyway
The unlock list was assembled by searching for applications that are blocked, which finds blocked applications. The honest test is the other one: sampling-shaped work that ships perfectly well today. If there is a lot of it, the argument above is weak. There is a lot of it, and looking at how it ships changes the claim rather than defeating it.
| Ships at scale | And it is genuinely sampling | How it ships |
|---|---|---|
| Real-time ray tracing | Monte Carlo integration over light paths, in every frame of a shipped game | At one to two samples per pixel, sometimes fewer than one, with a denoiser reconstructing the rest from neighbouring pixels and previous frames. It ships by taking the fewest samples physically possible and inferring the others. |
| MCMC in weather and statistics | Posterior sampling, the gold standard, in daily operational use | Its cost is called generally prohibitive. A likelihood evaluation means running the full weather model and MCMC wants thousands to millions of them, which is stated outright to be unrealistic, so a surrogate model stands in for the thing being sampled. |
| Language model inference | Sequential sampling, one token at a time, at planetary scale | Every step is a dense matrix multiply, which is the one operation a GPU was built for, and the sequential depth is amortised across a large batch. Nothing is removed, because nothing needs to be. |
| Neural speech and image generation | Iterative denoising, in consumer products | Distilled from many steps to one or a handful, which is a research literature in its own right, and run in a datacentre at a per-request price. |
Three of these four ship by not doing the sampling. One sample and a denoiser. A surrogate instead of the real likelihood. A thousand steps distilled into one. Each is a quality compromise accepted for a cost reason, and each would be abandoned tomorrow if sampling were cheap: nobody prefers 1spp with ghosting to 1000spp without it.
So the claim above is wrong as stated, and the correction is worth more than it was. The question is not which applications are blocked. It is which applications are paying a quality tax to avoid sampling, and that is a much larger set, because it includes everything on this list except the fourth row.
The fourth row is the one that matters, and it is the criterion. Language model inference is sequential sampling that nobody had to compromise, and the reason is that its per-step work is a dense matrix multiply. The GPU already ate that case. What it has not eaten is the per-step work that is not matmul-shaped: constraint propagation over a sparse, irregular graph, where the arithmetic is trivial and the access pattern is the cost. Ising sampling is that. Ray tracing is close to it, which is why it was compromised rather than scaled.
So the test an application must pass is not “does the loop sample”. It is does the loop sample, AND is the per-step work something other than a dense matrix multiply. The first question decides whether alternative hardware is relevant at all. The second decides whether the GPU has already won.
The second question, applied
Arguing for a criterion and then not applying it is how a thesis survives contact with its own data. So the 55 rows that scored samples or partly went back through the second question: twice, by two raters who did not see each other's calls, with every disagreement adjudicated separately. They agreed on 50 of 55. The five that split are marked • in the table above, and each turned on the same thing: whether you judge the shipped implementation or an expressible formulation of it.
Of the 40 rows scored fully sampling-shaped: 20 constraint, 20 matmul. The second question halves the list, and it does not cut where the thesis wanted.
The 20 that survive are one family, and it is not the exciting one: assign, schedule, route, place, partition. Batch scheduling, changeover sequencing, warehouse pick-paths, vehicle routing with time windows, charging stops, parking assignment, operating-theatre scheduling, radiotherapy planning, trial-arm balancing, matchmaking, level generation, crowd avoidance, contact resolution, room layout. Sparse graphs, discrete assignments, moves whose next step depends on the last. Exactly what an Ising machine is for, and exactly what a GPU is bad at.
The 20 that fall away include this page's own headline case. “Real-time world generation” was written here as the case: constrained, interactive, the user waiting. Both raters called it matmul: what actually streams the next region is a seeded noise-field evaluation or, in the systems being bet on now, an autoregressive video-diffusion forward pass per frame, and the coherence problem is being attacked with bigger models and longer context rather than with a solver. Neural codec decode went the same way, and more pointedly: shipped neural codecs deliberately dropped serial autoregressive context models for parallel chunked entropy coding, precisely because the serial part was the bottleneck. Those two were the Institute's own game-engine and media-codec explorations. The criterion says the GPU has them.
Which leaves a tension with the unlock list above, worth stating rather than smoothing. All four unlock-list applications (diffusion policies, on-device generative imaging, live generative content, conformer generation) are matmul-shaped per step, because a denoising step is a neural forward pass. By the criterion just argued, the GPU has already won them.
The resolution is that there are two different arguments for alternative hardware on this page, and the second question only tests one of them:
Structural
The per-step work is not matmul-shaped, so a GPU is the wrong machine for it and always was. The evidence is what the algorithm is. 20 use cases.
Substitution
The per-step work is matmul-shaped and the GPU does it correctly, but a physical sampler could do the same job for far less. The evidence is a simulated projection. The four on the unlock list.
These are not the same claim and they do not carry the same weight. The structural one is about the shape of the computation and needs no new hardware to be true. The substitution one rests on denoising thermodynamic models, replacing each neural denoising step with a Gibbs sample from a hardware energy-based model, whose headline figure of GPU parity at about ten thousand times less energy comes from simulation on a small-scale image benchmark, and is graded a projection on this page for exactly that reason.
So the honest summary is narrower than the one this page opened with. Where the case is structural it is 20 use cases of scheduling, routing, assignment and placement: unglamorous, present in every domain, and genuinely outside what a GPU is built to do. Where the case is substitution it is four applications with a real mechanism and an unmeasured magnitude. Both are worth working on. Only one of them is settled by knowing what the algorithm looks like.