Silicon Rivalry: Meta Deploys Google TPUs in Production via New PyTorch Integration

Google’s tensor processing units (TPUs) are now running in production environments at Meta, the developer of the widely used PyTorch framework. This deployment is enabled by a newly announced PyTorch-native version of the SGLang inference engine, detailed in a recent post on Google’s developer blog. Simultaneously, an open-source counterpart for vLLM, another major inference engine, was released. These updates allow the hardware to function as a native option for PyTorch workloads, largely eliminating the need for extensive code rewrites that were previously required.

Three key developments define this shift. First, Meta’s PyTorch-TPU lead, Zongwei Zhou, confirmed that SGLang-TorchTPU is actively serving AI models in production. He described the achievement as the result of a close three-way collaboration between Meta, Google, and the SGLang team. Second, Jana van Greunen, director of PyTorch engineering at Meta, stated that the company cannot be constrained by a single hardware architecture. She noted that the new TorchTPU integration enables the deployment of PyTorch workloads directly onto Google’s TPUs. Third, the announcement follows closely on the heels of Amazon’s decision to open-source TorchNeuron, its native PyTorch backend for Trainium2 and Trainium3 chips, at the same PyTorch Conference.

The move signals a strategic departure for Meta, which previously relied on Google’s PyTorch/XLA as a bridge to TPUs. Google has characterized PyTorch/XLA as a predecessor to the new TorchTPU. Industry reports indicated that Meta research teams utilized TPUs through this older bridge between 2020 and 2023 before canceling those contracts. The current push by Google to support PyTorch on TPU was primarily aimed at accommodating Meta, which had resisted migrating to Google’s JAX framework. While the precise location of Meta’s production TPUs remains undisclosed, reports suggest negotiations last year included Google Cloud rentals for 2026 and potential purchases in 2027, culminating in a multi-year rental agreement.

Performance benchmarks suggest the new integration is competitive with industry standards. A tuned vLLM setup serving Moonshot AI’s open-weight Kimi K3 model reportedly exceeded 700 tokens per second for a single user on the seventh-generation Ironwood TPU. This figure was provided by Inferact, the startup founded by vLLM’s creators, which assisted Google in building the new engine. The test utilized 16 Ironwood chips with speculative decoding, a technique that accelerates large language model inference using a smaller draft model. In comparison, vLLM’s published setup on 16 Nvidia GB200s achieved 452 tokens per second.

For smaller batch sizes without speculative decoding, the new kernels for Kimi K3 and Qwen 3.8 27B deliver throughput roughly 1.4 to 2 times that of the GB200 baseline. SGLang-TorchTPU is scheduled to support additional models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk, the startup founded by SGLang’s creators where Zhou’s team works, aims to provide Day 0 TPU support for new open models by the first quarter of 2027. This ensures compatibility on the release date of new software.

The convergence of these technologies appears designed to loosen the software lock-in traditionally associated with Nvidia. Both Meta’s and Amazon’s moves highlight a broader industry trend toward diversifying hardware options for AI inference. Google, Meta, and RadixArk plan to share further details at the SGLang Summit 2026, scheduled for November 12-13 in San Francisco.