Google open-sources TorchTPU, a native PyTorch back end for TPUs

You can now run PyTorch models on Google's TPUs with greater ease, potentially reducing training costs and accelerating development.

Google has open-sourced TorchTPU, a project that provides a native PyTorch backend for its Tensor Processing Units (TPUs). This release, available on GitHub, includes custom ATen kernels, a compilation cache, and support for distributed training, aiming to simplify the process of running PyTorch models on TPU hardware. The project also offers examples for various popular models and integrates with PyTorch's `torch.compile()` for graph execution. Installation can be done via pip with a specific extra index URL for PyTorch CPU versions or by building from source using Bazel. The project also details setup for VSCode with clangd for C++ development and outlines contribution guidelines.

The open-sourcing of TorchTPU signals a broader trend of hardware vendors providing first-party software integrations for popular AI frameworks. This move by Google aims to democratize access to its specialized TPU hardware, making it more competitive against NVIDIA's GPUs by offering a more seamless PyTorch experience. It lowers the barrier to entry for researchers and developers who prefer PyTorch but want to leverage the performance and efficiency of TPUs for large-scale model training and inference.

This development directly impacts the AI hardware ecosystem, potentially increasing TPU adoption and challenging the dominance of GPU-based solutions for certain workloads. Developers working with large language models or other computationally intensive AI tasks now have a more robust and officially supported path to utilizing TPUs, which could lead to faster iteration cycles and more cost-effective deployments. The focus on integration with `torch.compile()` suggests a commitment to leveraging PyTorch's latest performance optimization features.