After Google had already built specific Tensor Processing Units for training and inference, respectively, it has now delved even deeper into the bottlenecks of agentic workloads. The 8tx offers the most secure inference possible, while the 8i Max delivers massive context windows and high compute density.
The 8t (training) and 8i (inference) represented a giant leap forward in terms of efficiency and performance back in April of this year. Google is now staying within the same generation but sees room for further specialization. The 8tx is fundamentally more secure thanks to confidential computing. Despite the “t,” training isn’t the priority, at least not on a massive scale. Inference and compact training runs are the best fit for this chip, where keeping proprietary models private is of the utmost importance. With the 8tx, data remains encrypted in memory, the biggest gap in the security chain for AI training and inference data. The 8tx is paired with fifth-generation AMD EPYC CPU hosts.
The 8tx uses the Boardfly network that Google developed for the 8i. This results in up to half the latency compared to the TPU 8t. As a result, organizations can fine-tune models with their own data, distill large models, and train agents via reinforcement learning, all on a single system. Sensitive data does not need to be moved between clusters for this purpose.
8i Max: Google’s Blackwell?
A key difference between Nvidia’s Blackwell generation and its predecessors is that the B200 and GB200 Superchip represented a sum of their parts: two pieces of GPU silicon connected via the substrate at an extremely high speed. This effectively resulted in a single chip that was larger than what production methods allow for cost-effectively. Google is taking the same approach with the 8i Max, the most powerful TPU thanks to this combination.
Google is targeting a persistent problem with AI chips: the so-called “memory wall.” No matter how fast your GPU or TPU is, memory bandwidth limits performance once certain hard limits are reached. Consider a model that doesn’t fit into the memory of a single AI chip; communication between processors is many times slower than interaction within a single chip. The 8i Max doubles the computing power of the 8i, while also doubling the ultra-fast SRAM and HBM bandwidth. All of this is also more compact than standard 8i racks, with twice the silicon for the same rack space.
Software must lower the barrier to entry
In addition to the hardware, Google is also addressing the software. Last month, Google made TorchTPU open source, allowing PyTorch code and Hugging Face models to run natively on TPUs. Among other things, TPU Sync is new, offering an open-source layer that combines HBM, host DRAM, storage, and the network into a single memory hierarchy. According to Google, this reduces the Time to First Token by more than 90 percent. Cloud Pathways and GKE dynamic slicing are now generally available. With dynamic slicing, jobs start up up to five times faster.
Customers can also deploy TPUs in their own data centers. Additionally, Oracle Cloud Infrastructure will begin offering TPUs starting in 2027. According to Business Insider, Anthropic has also committed to purchasing up to one million TPUs, and Meta is using the chips for model serving.