Thermodynamic compute in gaming and media.
Gibbs sampling was invented in 1984 to restore images, and by 2000 graphics had put it down and picked up faster deterministic methods instead. The reasons were written down at the time. This topic studies which of those reasons were about speed, which were about the quality of the answer, and what happens to each when the same operation runs about ten billion times faster than it did when the decision was made.
1.06 × 10¹¹
categorical site-updates / s
64³ lattice, K=6 classes, 6 neighbours, browser WebGPU. Apple M5 Max, 40-core GPU, headed Chrome. 4096 sweeps per frame at 98 fps, flat across a further 2×, so a ceiling rather than a vsync artifact. This is a top-end laptop part, not a typical machine.
0.17 ms
to equilibrate one 64³ region
70 sweeps to plateau at the rate above. About 92 such regions fit in a 16 ms frame, which is why a world can be re-melted while you stand in it.
97.7 %
of crystal ended up in the lower half
No layer was authored. Stratification is what one energy function does under a gravity term, measured over the sampled field.
27.0 dB
PSNR of doing nothing at all
Holding the previous frame, 0 bytes. The lab's own temporal codec scores 10.9 dB at 416 bytes per frame. See the negative results below.
One operation, explained once.
Everything on this page is the same computation. It is worth understanding before running any of it, because the instruments are only interesting if you can see what they are doing.
Take a grid. Every cell holds one label out of a small set: stone, ore, air, water. Write down an energy that only ever looks at pairs of touching cells, giving a low number to pairs that belong together and a high number to pairs that do not. That single function defines a probability for every possible arrangement of the whole grid, with low-energy arrangements more likely, and a temperature controlling how strongly that preference is felt.
To draw an arrangement from that probability, visit one cell, look only at its neighbours, and pick a new label with probability proportional to how well it fits them. Repeat. That is Gibbs sampling, and it is the whole method. At high temperature the grid stays disordered; lower the temperature and regions of agreement grow. Add a term that makes heavy labels prefer to be low, and the grid will stratify on its own, with no one having drawn a layer.
Two more ideas complete the picture. Clamping a cell means fixing its label and letting everything else arrange itself around that constraint. That one mechanism does three jobs here: it is how a mined tunnel stays open, how a codec's known pixels constrain the unknown ones, and how a designer states an intention. And a seed plus the rules is enough to reproduce an entire world, which is why a session in the game below travels as a few dozen bytes.
The field tried it, and left.
This is not a new idea being proposed. It is an old idea being re-examined, and the honest way to teach it is to say plainly why it was set aside.
Stuart Geman and Donald Geman introduced Gibbs sampling in 1984 for image restoration, describing it as a highly parallel relaxation algorithm. Texture came a year earlier: Cross and Jain fitted Markov random fields to real textures in 1983, and found that microtextures fitted well while regular and inhomogeneous ones did not. The approach reached its high point with FRAME in 1998, which matched filter statistics and used a Gibbs sampler to draw the images.
In 1999 Efros and Leung gave two reasons for moving on, and the difference between them matters more than anything else on this page. The first is about cost:
"Gibbs sampling is notoriously slow and in fact it is not possible to assess when it has converged."
The second is about whether sampling helps at all. They ran a Gibbs sampler on top of their own much simpler greedy result to see if it improved:
"our trials have shown that Gibbs sampling produced very little improvement for most textures. This lack of improvement indicates that the heuristic indeed provides a good approximation to the desired conditional pdf."
Everything after that keeps the energy and discards the sampler. Wei and Levoy searched deterministically and ran two orders of magnitude faster. Image quilting found a minimum-cost seam by dynamic programming. Graphcut Textures took the same energy on the same sparse graph and solved for the most likely arrangement instead of drawing one, which min-cut does exactly. Lefebvre and Hoppe reached real time on the GPU with no temperature anywhere in the method. PatchMatch, which became Content-Aware Fill, searches rather than samples.
Speed is the reason that has changed. The other one has not, and it is a statement about the quality of the answer rather than the time taken: running a slow method faster arrives at the same conclusion sooner. Any claim this topic makes has to face the second quotation, not the first.
Three instruments, one kernel.
Each page below samples a categorical field on a sparse graph at a set temperature, given clamps, reproducibly from a seed. They differ only in the graph and the clamp pattern, and they run entirely on your own device.
A note on how these behave. Each field settles to equilibrium and then stops. A settled field is not resampled to draw it, because another sweep would produce a different draw from the same distribution: every cell would change, and none of the change would carry information. Sweeps resume only where the field has genuinely been disturbed.
PLAYABLE
A world you mine, not a mesh you load
First person, on a 64³ field that was annealed rather than authored. The stratification, the ore pockets and the water table are all consequences of one energy function under gravity. Blasting detaches cheapest bond first, so a charge follows the microstructure the sampler grew instead of carving a sphere. Mined-out air is clamped, which is the same mechanism the codec uses for its scaffold, and it is why a tunnel does not quietly refill behind you.
CODEC
Transmit the conditions, thermalize the rest
Here the decoder is the sampler. The wire carries a set of couplings fitted from the content, a schedule, and a sparse scaffold of clamped pixels; the receiver fills in everything between them by sampling. Four sources are offered on purpose. Two of them it handles, two of them it does not, and the boundary between those pairs is more informative than either.
KERNEL
The kernel by itself, with its rate on screen
The same chromatic block-Gibbs pass with no game on top. Press measure to take a timed window and read the site-updates per second your own machine reaches. The number in the table above came from this page, and it is worth confirming rather than believing.
Reported at the same size as what did.
Four results came out against the idea. Each is reproducible in the instruments above, and the last one is the most useful thing on this page.
The temporal codec loses to doing nothing
Across 48 frames the codec sends 416 bytes per frame and reaches 10.9 dB. Emitting a single flat colour costs about one byte and reaches 17.1 dB. Simply repeating the previous frame costs nothing and reaches 27.0 dB. The codec is beaten by sixteen decibels by a decoder that does not run. Until it clears that line there is no compression result here, and the earlier figure of 33× should be read as a bitrate with no quality axis attached to it.
A fresh draw is not the same picture
The reason is worth more than the result. A sampler asked to reproduce a frame returns a different arrangement from a similar distribution, and a pixel-difference metric charges full price for every cell that moved even when the texture is statistically identical. That is the same fact that makes an equilibrated field wrong to redraw. It also means PSNR cannot referee this method, and no honest claim can be made until a metric is chosen that measures what the method is actually preserving.
The still codec loses to PNG on structured scenes
On a scene of straight edges and flat regions, at the scaffold density where reconstruction is faithful, the wire is 0.47× the size of a lossless PNG. A model that only ever looks at touching pairs has no way to represent the fact that a line continues. This is the 1983 finding about regular textures, reached again from a different direction.
84% agreement on unreadable text
On a page of text, class agreement reads 84% because the background dominates and is easy. The reconstruction cannot be read. Agreement is the wrong statistic whenever the content of interest occupies a small fraction of the field, and reporting it by itself would have been misleading.
Reading the numbers carefully.
- The throughput figure is from one machine and one vendor: an Apple M5 Max with a 40-core GPU. It is a fast laptop part rather than a typical one, so it supports the claim that the operation is fast on current hardware and does not support any claim about what most people own.
- The sampler runs a fixed number of sweeps rather than a convergence test, so what it produces is a bounded-budget approximation rather than a guaranteed draw from the equilibrium distribution. The 1984 objection about not knowing when a chain has converged applies to this work exactly as written.
- There is floating-point arithmetic on the accept path, including an exponential, and WGSL does not require identical results across vendors. Reproducing a world from a seed is therefore reliable on one implementation and not yet demonstrated across them. Removing the float is a prerequisite for treating a seed as a contract.
- The couplings in the codec are fitted from the content being sent. Those fitted values are part of the message, and any compression ratio quoted without them is incomplete.
- No joules are reported anywhere on this page. The lab's standing measure is energy per task, and applying it to this work is outstanding.
In order of cost to decide.
- Which metric should referee a generative codec? Until this is settled the temporal result above cannot be improved, only re-measured against the wrong ruler.
- Efros and Leung found that sampling added almost nothing on top of a good greedy result. Does that still hold when the sampler is ten billion times faster, or was the finding about the estimator rather than the budget? This is the question the whole topic rests on.
- Which content has the tail? The codec wins on one class of image and loses on another. A two-point spread is not a distribution, and the useful result is the boundary rather than either end.
- How large is the gap between greedy per-block rate-distortion optimisation in a modern video encoder and a joint optimum over the same choices? If it is small, an encoder built on sampling has nothing to win.
- AV1 already transmits film-grain parameters and resynthesises the grain at the receiver. Games have shipped seed-plus-rules reproduction to a far larger audience. How far does that contract generalise, and what breaks it first?
- At what energy per sample does dedicated hardware earn its place against a GPU that is already this fast and comes with a complete toolchain? The throughput result makes this case harder to argue, not easier.
ferrotherm.
Pure Rust, no dependencies, wasm-clean, reproducible from a seed. Sparse Ising and Potts models with chromatic block-Gibbs, parallel tempering, thermodynamic linear algebra, a compiler that fits programs onto device topologies, and a ledger that meters every sample, read and write against swappable device models.
Onsager's closed form, exact-Boltzmann total-variation distance, enumerated ground states and Gaussian elimination all sit in the test suite as referees, so the sampler is checked against answers that are known independently of it. Every claim carries a grade, measured or verified or projected, and the grade travels with the claim.
The library ↗Thermodynamic Physical AI →How we measure energy (PAI-280) →
Sources, in order.
Quoted material was taken from the source text. Where a paper could not be retrieved in full it is listed for its bibliographic record and is not quoted.
- 1983 · Cross & Jain. Markov random field texture models. IEEE TPAMI. The first fit of MRFs to real textures; microtextures worked, regular and inhomogeneous ones did not.
- 1984 · Geman & Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE TPAMI 6(6). Where the sampler comes from.
- 1987 · Marroquin, Mitter & Poggio. Probabilistic solution of ill-posed problems in computational vision. Introduces the MPM estimator and argues it beats MAP for restoration.
- 1995 · Bader, JáJá & Chellappa. Scalable data parallel algorithms for texture synthesis using Gibbs random fields. IEEE TIP 4(10). Massively parallel Gibbs on a lattice, used for both synthesis and compression.
- 1998 · Zhu, Wu & Mumford. Filters, random fields and maximum entropy (FRAME). IJCV. The high-water mark of sampled texture.
- 1999 · Efros & Leung. Texture synthesis by non-parametric sampling. ICCV. States both reasons the field moved on.
- 2000 · Wei & Levoy. Fast texture synthesis using tree-structured vector quantization. SIGGRAPH. Keeps the MRF model, replaces the sampler with a deterministic search.
- 2001 · Efros & Freeman. Image quilting for texture synthesis and transfer. SIGGRAPH. Minimum-cost path by dynamic programming.
- 2003 · Kwatra et al.. Graphcut textures. SIGGRAPH. Same energy, same sparse graph, argmin instead of a sample.
- 2005 · Lefebvre & Hoppe. Parallel controllable texture synthesis. SIGGRAPH. Real-time on the GPU and explicitly deterministic; variation comes from jittered exemplar coordinates rather than a temperature.
- 2008 · Jalali & Weissman. Rate-distortion via Markov chain Monte Carlo. Sampling as the compression primitive, with an optimality result for stationary ergodic sources.
- 2009 · Barnes et al.. PatchMatch. Randomized search over a correspondence field, converging to a fixed point. Ships as Content-Aware Fill.
- 2015 · Versteegen, Gimel'farb & Riddle. Texture modelling with nested high-order MRFs. Records that Gibbs sampling from these models adds unwanted high-frequency noise, and that the Markov blankets had grown to 50–100 pixels.