Google open-sources TorchTPU, making Cloud TPUs a native PyTorch device

Google has open-sourced TorchTPU, its native PyTorch stack for TPUs, announced at PyTorch Conference 2026. The pitch is that TPUs now sit next to GPUs and CPUs as an ordinary PyTorch target: you debug in eager mode, then move to torch.compile for production speed, with PyTorch’s distributed primitives working as usual. Google says it was co-developed with the PyTorch team at Meta. The repository carries an Apache 2.0 license file.

It lands a day after AWS put out TorchNeuron for Trainium, so AWS and Google both pitched “keep your PyTorch code” for their own chips in the same week.

What comes with it

• Hugging Face. In Transformers v5, Google says deploying to a TPU is device_map="tpu", and Diffusers runs on TPUs with eager mode, torch.compile and tensor parallelism. Google says it runs the Transformers test suite on TPUs nightly.
• Serving. A native PyTorch vLLM backend built with Inferact, with custom Pallas kernels, continuous batching and quantization, and an SGLang backend built with RadixArk. Both keep the upstream schedulers and API servers. vLLM and SGLang already ran on TPUs through JAX backends; the Google Developers post says developers wanted to debug in PyTorch, and that the JAX versions stay maintained. A Meta engineer quoted there says SGLang-TorchTPU runs in production at Meta. On Kubernetes, the llm-d 1.0 router can front GPUs and TPUs with one endpoint on GKE.
• Training. TorchTitan recipes for dense, mixture-of-experts and diffusion models, a TPU backend upstreamed to the VeRL reinforcement-learning framework (in preview), and native support in Ray for mixed CPU, GPU and TPU pipelines.
• Edge. LiteRT, formerly TensorFlow Lite, joins the PyTorch ecosystem with a one-call litert-torch.convert().
• A TPU agent suite, in early preview on GitHub, which plugs into coding agents to help with onboarding, diagnostics and inference tuning.

The numbers, and whose they are

Google says TorchTitan on TorchTPU reached up to 52% model FLOPs utilization on Qwen 3 8B, with near-linear scaling up to 8,192 Ironwood TPUs. Partner quotes from Meta, Hugging Face, Inferact, RadixArk, Anyscale and Zenteiq (which cites more than 4x throughput through GKE Inference Gateway) are endorsements, not benchmarks you can rerun. Measure your own model before moving a budget.