November 22, 2026

Google Expands AI Infrastructure with Specialized TPU 8 Variants for Training and Inference

However, training is only one side of the AI coin. As models move from the laboratory to the real world, the focus shifts to inference—the process of using a trained model to generate responses or perform tasks. This is where the TPU 8tx comes in. While it retains the core compute power of the 8t, the 8tx is optimized for the low-latency requirements of inference, particularly for "agentic" workloads where multiple inference streams may be running concurrently to perform complex, multi-step tasks.

Google Rejiggers TPU Accelerators For Broader AI Inference Work

The most notable change in the TPU 8tx is the replacement of the 3D torus interconnect with a "boardfly" topology. This is a variant of the dragonfly interconnect designed by Google to create a more tightly coupled system. By reducing the pod size from 9,600 accelerators to 1,152, Google has effectively flattened the network. This reduction in the fabric’s diameter results in an average latency that is roughly half that of the 3D torus configuration.

Furthermore, the TPU 8tx introduces a surprising shift in the host processor. Google has opted to replace its own Arm-based Axion CPU with an unnamed AMD Epyc processor for this specific variant. This decision appears to be driven by security requirements. The AMD Epyc chips feature specialized on-board root of trust and encryption accelerators. This allows data to remain encrypted until the exact moment it is processed by the TPU, and to be re-encrypted immediately afterward. While Google has its own security processors, the integration of these features into the AMD silicon provides a ready-made solution for high-security inference tasks. Industry observers speculate whether this move to AMD is a temporary measure until Google can integrate similar encryption features into its next generation of Axion and TPU 9 architectures.

On the more efficiency-focused side of the spectrum is the TPU 8i Max (Zebrafish). This variant is designed to tackle large-context AI inference, where the model needs to "remember" or process vast amounts of information in a single session. To achieve this, Google has doubled the number of Zebrafish accelerator complexes per socket. However, the memory configuration reveals a complex set of architectural trade-offs.

While the original Zebrafish 8i chip featured 288 GB of HBM3 memory, the TPU 8i Max utilizes two chips, each with only 192 GB of capacity. Although the total memory per socket has increased, the amount of memory per unit of compute has actually decreased by about 33 percent. This suggests that Google is spreading its HBM3 allocation across more chips to create a larger aggregate shared memory footprint for things like Key-Value (KV) caches, which are vital for maintaining context in long conversations. This approach risks leaving some compute power "stranded"—meaning the chip has more processing ability than it has memory to support—but for large-context inference, memory capacity is often the more critical bottleneck.