❯ Google unveils TPU 8tx and 8i Max to tackle hardware bottlenecks in confidential inference and long-context AI

Four systems in one TPU generationOn November 9, Google Cloud announced two new TPU systems. Google designed these accelerators for AI computing, initially for its own workloads and now also for cloud customers. In April it announced the 8t for large-scale training and the 8i for real-time inference. The 8tx and 8i Max address sensitive data and long inputs, respectively. Google infrastructure chief Amin Vahdat told Bloomberg that both systems will be available in 2027.

8tx extends protection to data in useSensitive information can be encrypted in storage and in transit, but a model still needs to access it to do its work. The 8tx combines 8t silicon with fifth-generation AMD EPYC host processors and isolates data and model weights in hardware while they are in use. Customers with strict compliance requirements can run a proprietary model and fine-tune it on a small scale with internal data on the same system, reducing the need to move that data between clusters.

8i Max packages two chips togetherWhen models read an entire codebase, hours of video, or records from multiple agents, feeding that information to the processors can become the bottleneck. Google says putting two 8i chips in a single package doubles compute, on-chip static memory, and high-bandwidth memory throughput relative to the 8i. Google also says the system can fit more computing hardware into the same rack footprint, useful for services that routinely process long inputs.

Customer workloads now shape the hardwareGoogle says TPUs have nearly 100 customers, and it has started letting customers deploy them in their own data centers. The 8tx and 8i Max respond to separate requests for stronger data isolation and larger context windows. Reusing the underlying silicon and changing the package, network, or host helps Google deliver a specialized system sooner than designing a new chip for each workload.

▮ SIGNALGoogle is increasingly competing with whole TPU systems, where customers buy the combination of security, memory bandwidth, and rack efficiency that their workloads need.