Sonos's Rust inference engine tract now runs Stable Diffusion 1.5 in the browser through WebGPU

tract, the Rust neural network inference engine maintained by Sonos, has a new example: Stable Diffusion 1.5 running in the browser, on the visitor’s own GPU. It went into tract’s main branch on 27 October and is live as a demo.

How it works

According to the README, the page downloads SD 1.5’s fp16 weights (2.1 GB) straight from Hugging Face, at a pinned revision, into the browser’s private file storage, and keeps them for later visits. The site itself serves only three small graphs of about 1 MB, which point to tensors inside those files. “Nothing leaves the machine.”

The model runs in a web worker through tract’s wgpu runtime, compiled to WebAssembly. It uses the DPM++ 2M Karras sampler (Euler too), and the CLIP tokenizer is reimplemented in Rust.

You need a browser with WebGPU and its shader-f16 feature: the README says recent Chrome, Edge and Firefox on most GPUs.

Not on crates.io yet

The example uses features flagged unstable, such as async GPU readback and linking named weights from safetensors files, and it builds against tract’s main branch. The latest crates.io release is still 0.23.9, which we covered in our tract 0.23.9 story. Expect these pieces in a future release.