Google Expands AI Chip Portfolio with TPU 8tx and TPU 8i Max

Google announced last Monday, November 10, two new systems based on its eighth-generation Tensor Processing Units (TPUs), marking the fourth hardware expansion by the company in just one year. The TPU 8tx and TPU 8i Max are not physically new chips — they are systems designed around existing architecture, but with distinct specializations to meet emerging demands from autonomous AI agents.

The TPU 8tx was designed specifically for confidential inference environments, with hardware-level security for sensitive data. The TPU 8i Max, meanwhile, stacks two chips in a single package, achieving the highest compute density per rack in the family. Together, they join the already-launched TPU 8t (large-scale training) and TPU 8i (real-time inference), completing a portfolio of four systems in a single generation.

What changes in the architecture

The key difference between a "new chip" and a "new system" is the focus of the design. Designing a chip means arranging billions of transistors on a silicon wafer — a two-to-three-year process with foundries like TSMC. A system, on the other hand, is about how to organize compute, HBM memory, optical networking, and storage around the chip. Google emphasizes that this modular approach enables delivering specialized hardware configurations in months, not years.

The TPU 8tx introduces specialized encryption capabilities for highly regulated workloads — governments and financial sectors that need to run proprietary models on private data without exposing sensitive information to cloud providers. It is the same reasoning that led Apple to develop the Neural Engine for on-device processing, now scaled up to enterprise data centers.

The TPU 8i Max, for its part, solves a fundamental bottleneck of AI agents: memory. When an agent needs to reason over long documents — entire contracts, thousands of chat messages, hours of transcripts — it needs to keep those contexts in the chip's memory. The dual packaging of the 8i Max drastically reduces inter-chip communication latencies, cutting delays by up to 50% compared to the previous generation.

The strategy behind the expansion

The decision to launch four systems in a single generation reflects a deliberate strategy. According to Amin Vahdat, Google's CTO for AI and Infrastructure, the company has been receiving growing external feedback on specific hardware needs. "With compute moving from human scale to agent scale, businesses are asking for accelerators that can reason across massive contexts," Vahdat said on Google's official blog.

The AI chip market is valued at hundreds of billions of dollars, and Google wants a significant slice. With nearly 100 customers — including frontier AI labs, trading firms, universities, and retailers — the company is attempting to replicate Nvidia's success on its own turf. Unlike Nvidia, which sells general-purpose GPUs for multiple workloads, Google offers hardware specialized for specific stages of the AI lifecycle.

Software and ecosystem

The systems come with software stack updates. Google introduced the XLA Serving Engine, an open-source component that allows training models in Python and serving them directly in a low-latency C++ runtime. In production benchmarks (InferenceX and AgentX), the native vLLM-TorchTPU stack on TPUs achieved state-of-the-art price-performance marks for agentic workloads and long contexts.

The Agent Compute Engine, another component of the ecosystem, automatically suspends and resumes idle agent sessions — reducing operational infrastructure costs by detecting when an agent has been inactive for a certain period. It is a feature that reflects the paradigm shift: when thousands of agents execute simultaneously, resource efficiency becomes as critical as raw speed.

Google also made GKE Dynamic Slicing available, which uses optical circuit switching (OCS) to dynamically assemble TPU topologies — splitting or combining physical clusters into logical slices on demand. This reduces job startup times by up to 5x and accelerates fault recovery by up to 4.5x.

The Nvidia challenge

The rivalry with Nvidia is heating up. While Nvidia dominates the market with its H200 and H100 GPUs, Google positions the TPUs as the most efficient alternative for specific workloads — especially large-scale inference and autonomous agents. Bloomberg reports that Google is scaling its specialized chip strategy precisely to compete directly with Nvidia in a market that should exceed $300 billion by 2027.

The strategy works up to a point because Google builds TPU systems before even having paying customers — a risky model that nevertheless allows the company to maintain innovation speed superior to typical foundry cycles. If the current portfolio can retain its 100 customers and expand to 150 or 200, Google will have created the second most relevant force in AI hardware — after, naturally, Nvidia itself.

The question that remains is whether extreme specialization across eight TPU generations will convince enterprises to migrate from conventional GPUs to TPUs — or whether ecosystem fragmentation will keep Nvidia as the de facto standard. What is certain is that the race for AI hardware has entered its most complex phase: it is no longer enough to be fast. You must be specialized, secure, and integrated with a software stack that actually works.

Sources: Google Cloud Blog, Bloomberg, Forbes

✓ Independent sources cross-checked and verified before publishing