As AI shifts from being a chatbot to being an autonomous agent, compute costs start to dominate the conversation. That's the point Nvidia is betting on this Tuesday (August 11): the company expanded the Nemotron 3 family with Nemotron 3.5 Lightning, an open 30-billion-parameter model built on a mixture-of-experts (MoE) architecture, and released NeMo Switchyard, an open source library for "routing" each call to the most suitable model.
The idea is to attack one of the most expensive problems of modern agents. "Always-on" autonomous systems increasingly work as ensembles of models: a frontier reasoning model — such as Nemotron 3 Ultra or GPT-5.6 — plans and orchestrates the workflow, while smaller specialized models execute specific tasks like code review, tool use, security alert monitoring, or answering billing questions. The bottleneck is that using the giant model for everything is slow and costly.
That's where 3.5 Lightning comes in. According to Nvidia, it delivers up to 4x faster output speed and 30% faster agentic task completion compared with other models in its class, while maintaining frontier-level accuracy on PinchBench benchmarks. And because it's open and customizable, it can be post-trained with NeMo on an organization's own data, tools, and workflows.
Switchyard, in turn, is the piece that ties it all together. It's an intelligent routing library that decides, for each request, which model in the mix — open, proprietary, or Nvidia's own — is the most capable and efficient for the task, without requiring developers to rewrite their applications. In practice, it's like a traffic controller that sends each vehicle to the right lane instead of letting everyone cross the same highway.
The early adopters hint at where the company wants to go: CrowdStrike is customizing the model for cybersecurity, Harvey (with Trajectory) for legal services, and CodeRabbit (with Baseten) for code review. These are high-volume use cases, where cost per call and latency matter more than raw peak power.
Nvidia's move has a clear strategic reading. The company built its fortune selling the chips that train the world's largest models, but the inference market — running those models day to day — is where long-term demand tends to concentrate. At the same time it sells the hardware to train the frontier model, it offers the software and the small model that make agent operations cheaper. It's a bet across the whole cycle, not just the top.
There is, however, a tension that becomes obvious. If routers like Switchyard spread the practice of using small models for most tasks and reserving giants for what truly requires them, demand for top-tier GPUs for inference could be lower than the market prices in. Nvidia, apparently, would rather cannibalize part of that growth than lose the agents race to competitors with more efficient open models.
What the launch reveals, in the end, is the industry's new consensus: the future of agents does not belong to a single omnipotent model, but to systems of models that trade intelligence for efficiency in real time. For enterprises, the news is good — it means cheaper, more scalable agents. For the chip market, the uncomfortable question is how much of that future Nvidia will dominate — and how much of it the company itself will help compress.
Sources: NVIDIA Blog, Quartz, Hugging Face
✓ Independent sources cross-checked and verified before publishing
