The race for AI supremacy gained a quiet but significant new chapter over the weekend when a team of engineers at Doubleword managed to run DeepSeek's V4 Flash model on a single AMD Instinct MI300X GPU. The story blew up on Hacker News, where the detailed technical write-up earned dozens of upvotes within hours. Calling it a milestone is not an exaggeration: we are talking about a mixture-of-experts (MoE) model with 284 billion parameters — even if only about 13 billion are active per token — running entirely within the memory of one accelerator, with no distributed cluster in sight.
The most revealing detail of the account is that, as of early May, this simply did not work. The standard vLLM inference stack would not load V4 Flash on the MI300X, and the engineers had to wade through a swamp of software incompatibilities. The main obstacle was the FP8 dialect: the MI300X, built on CDNA 3, implements AMD/Graphcore's own E4M3 variant, while the MI325X and newer boards follow the OCP standard. A kernel that assumes OCP semantics on the MI300X can be wrong by a factor of two in the scale domain — which, translated directly, means corrupted output. Optimized attention kernels were also missing, and the work turned into a war diary against the ecosystem's sharp edges. After beating each one in turn, the team reported around 2,699 tokens per second at inference.
Why does this matter? Because the MI300X was designed precisely for this kind of workload. It packs 192 GB of HBM3 memory, 5.3 TB/s of bandwidth, and 2.6 PFLOPS of FP8 compute into a single OAM module. Large language model inference is bound by memory bandwidth, not raw compute capacity. When a model fits entirely in one board's memory, costs collapse: you eliminate the dependence on high-speed interconnects between GPUs, on distributed scheduling, and on large pools of shared VRAM. That is the promise that has always made AMD an attractive alternative to Nvidia's hegemony — and it is now beginning to materialize outside marketing slides.
The broader implication is the democratization of hardware. If frontier-model inference can run on a single affordable board, the cost per token falls and the barrier to entry for smaller companies — and even individual developers — shrinks dramatically. Structurally, it also weakens the CUDA ecosystem lock-in: whoever runs V4 Flash on an MI300X demonstrates, in practice, that escaping Nvidia's software monopoly without sacrificing performance is possible. The community effort, including the hand-written kernels from the AgntroAI project that tripled single-stream decode speed from 20.7 to 58.9 tokens per second, is living proof that the momentum now comes from the ground up.
But the question left open is uncomfortable and honest: software is still the bottleneck. Every such triumph is earned through hard, manual work against ROCm's immaturity and FP8 dialect fragmentation. The hypothesis that AMD can rewrite the economics of AI depends entirely on hyperscalers and providers pouring real investment into that software stack. Is it worth their while to bet on kernels optimized for the MI300X, or will progress keep riding on hobbyists and enthusiasts? The answer will determine whether this milestone is the start of an era or just a heroic, isolated victory.
Sources: Doubleword (Fergus Finn), SimpleNews, 1ban News
✓ Independent sources cross-checked and verified before publishing
