--- skip_grammar_check: true skip_spelling_check: true ---
NVIDIA announced on Monday, August 24, that the Groq 3 LPX, its dedicated artificial intelligence inference accelerator, has entered full mass production. The announcement was made during Hot Chips, the annual conference for computer architecture researchers, and marks the culmination of an effort that began with the company's $20 billion acquisition of Groq's assets in December 2025. For the industry, the news confirms a strategic bet that splits AI inference into two layers: general-purpose GPUs for training and specialized silicon for the critical token-generation phase.
The Groq 3 LPX is an extension of NVIDIA's Vera Rubin platform, designed specifically to accelerate the "decode" phase — the moment when the model generates each token of its response. This is the phase that determines end-user perceived latency, especially in agentic systems that need continuous, real-time responses. While traditional NVIDIA GPUs excel at the "prefill" phase (analyzing the prompt), the LPX takes over next, ensuring token generation is deterministic and low-latency.
Each LP3 (Language Processing Unit) has only 500 MB of onboard memory — exactly 576 times less than a high-spec Rubin chip's 288 GB. This apparent limitation is not a weakness but a deliberate architectural choice. By focusing exclusively on inference, the chip eliminates circuits unnecessary for training and allocated almost all silicon area to high-speed buses and SRAM memory. Each LPX rack can accommodate up to 256 LPUs, totaling 128 GB of high-speed SRAM, enough to host models of up to 31 billion parameters in a single system.
The philosophy behind the Groq 3 LPX is that AI inference is not merely faster training — it is a fundamentally different problem. In the decode phase, each token depends on the previous one, creating a sequential dependency that GPUs do not optimize well. The LPX was designed to solve exactly this bottleneck, with a deterministic instruction pipeline inherited from Groq itself, which eliminates the need for complex buffers and branch predictions.
NVIDIA's purchase of Groq was, in the terms at the time, a "not-acqui-hire": NVIDIA acquired non-exclusive licensing rights to the inference technology, hired Groq's executive team, and absorbed its engineering staff. In June 2026, the remnant Groq raised $650 million from existing investors to continue operating independently, while NVIDIA began manufacturing and marketing the chips under its own brand. The strategy allowed NVIDIA to extend its product line without building an architecture from scratch — a process that would have required several more years of development.
Early benchmarks published by The Register reveal that the Groq 3 LPX can generate tokens at significantly higher rates than conventional GPUs in autonomous agent workloads. In tests with the Gemma 4 31B model, the LPX system demonstrated consistently low decode latency below 10 milliseconds per token, compared to the typical 25-40 ms of Tensor Core-based GPUs for the same task. The difference is even more pronounced in long-context sessions, where the overhead of conventional GPU memory management accumulates.
For NVIDIA, the Groq 3 LPX entering production is part of a broader strategy: building a complete "AI factory" where each pipeline component has a dedicated chip. Rubin GPUs for training, LPX for inference, MTIA for mass-scale inference, and now the NVL72 for energy efficiency. The vision is that as AI agents become the dominant interface between humans and computation, inference latency will be the primary competitive factor — just as training accuracy was five years ago.
The question remains whether this extreme specialization in inference chips will create a sustainable market for competitors or whether NVIDIA will consolidate the deployment layer so completely that the barrier to entry becomes insurmountable — and, in a scenario of such consolidation, what consequences that will have for innovation in the artificial intelligence ecosystem in coming years.
Sources: NVIDIA Newsroom, Quartz, The Register
✓ Independent sources cross-checked and verified before publishing
