Advancing AI 2026 is AMD’s effective launch party for Helios. This rack-scale system takes on Nvidia’s Vera Rubin NVL72, the current high watermark for AI training and inferencing at scale. Helios is AMD’s largest single compute unit for AI by a large margin, with full coherence across 31 terabytes of HBM4 memory. Apart from its raw specs, however, Helios is a tale of lessons learned and bottlenecks removed.
Critically, AMD has made Helios one coherent system. A pool of 31 terabytes (!) of HBM4 memory is addressable across the internal network fabric. The 72-GPU setup is thus on par with Nvidia’s rack-scale equivalent, albeit with 1.5x the memory. One Helios rack can serve multiple instances of frontier LLMs, dozens of large open-weight models and over a hundred LLMs at the common 70-ish billion-parameter count. For training, the memory headroom means fewer model replicas are required, leading to better tokens-per-dollar, as AMD puts it.
All 72 GPUs share a single load/store domain across the fabric. Any GPU can reach any other GPU in a single network hop. Software “no longer has to optimize for network locality,” as described by Mark Chubb, AMD’s Corporate VP of Platform Architecture. Or, to put it another way, software no longer needs to care where the data is.
Solving bottlenecks
The system also automatically routes around congestion, treating the 72-GPU aggregate as one coherent object. This means that many bottlenecks core to AI training and inferencing are removed, leaving programmers to optimize elsewhere. The scale-up bandwidth, i.e. the maximum bandwidth within the rack between GPUs, maxes out at 260 TB/s. Scale-out, so from rack to rack, sits at 43 TB/s.
Isolation layers let you partition Helios into independent workloads without data leakage. But the default posture is that of a single massive compute unit for AI. You can slice it up however you want, of course, and confidential computing allows customers to trust that critical workloads remain safe even when other organizations are running their AI on the exact same Helios system.
Another improvement comes from treating Helios as more than the servers inside racks. AMD’s “inside-out” thinking has combined with the expertise of acquired system integrator ZT Systems, providing a look at the practical requirements for making Helios easy to work with. A single compute tray weighs 77 kilograms and features a plethora of vulnerable components. Chubb, formerly of ZT Systems, suggests that AMD has gained a useful “outside-in” perspective for accounting for the physical nature of these systems.
The truth of the matter is that any competent enterprise GPU sells like hotcakes, and one reason AMD isn’t benefiting more is that it also has a server CPU and consumer business to run. Allocating wafers on TSMC’s 2-nanometer node will be tough, as both Epyc and Instinct (AMD’s AI GPUs) use it. Every Helios system requires 18 Epyc processors and 72 Instinct MI455X GPUs, stacked in 18 compute trays.
CUDA is (apparently) not a moat at all
AMD has had proper AI specs for years. However, practical factors have left it solidly behind Nvidia, even if we account for its gains in market share. Firstly, ROCm was not the standard and was a confusing software layer that couldn’t hold a candle to CUDA. In late 2024, it was reportedly “riddled with bugs.” Beyond that, repeatable failures damaged its potential for running immensely costly AI training runs. AMD was left to do inferencing.
Several factors are now helping AMD beyond the raw specs. For example, purchases are shifting towards inferencing over training, 60 percent to 40 percent. This split was inverted two years ago.
Second, and this will be considered contentious by many, CUDA is “not a moat,” according to AMD. The company’s Corporate Vice President and General Manager of the Data Center GPU Business Unit, Andrew Dieckmann, is blunt about it: “We’re almost having zero conversations about CUDA anymore.” Higher abstraction layers and automatic AI-assisted recipes for ROCm are helping to eliminate Nvidia’s CUDA stack as an important factor.
This would have sounded astounding three years ago. But there’s good reason to believe AMD is right. Take Anthropic, for example, which has managed to make its Claude models hardware-agnostic to a certain extent. Nvidia, AMD, Google TPU and AWS Trainium; they all run Claude, even Fable 5.
A company like Anthropic (or OpenAI, Meta, SpaceXAI and others, for that matter) has no reason to invest in a CUDA-centred world. AMD is helping them along. ROCm has improved tremendously, delivering inference optimization that is broadly competitive with Nvidia.
Also read: AMD to invest up to 5 billion in Anthropic for Instinct GPUs
Conclusion: the new inferencing reality
It’s odd to note that AMD is now seemingly competitive on training just as inferencing has clearly become the priority. In a sense, it missed the heyday of training-first build-outs from 2023 to 2025. But the scale of AI will only increase, and the need for competitive inferencing is fierce.
Dieckmann notes that there is an “enormous market” for AI workloads, with “room for compute diversity.” AMD’s specific role centres on flexibility. “Modularity is our friend,” he says, and AMD has many pieces of the AI puzzle. ZT Systems’ integration is one such component, as are the company’s leading Epyc CPUs as host nodes for GPUs and more, along with the Instinct AI accelerators, DPUs, NICs and fabric.
That compute diversity is already taking shape. Cerebras is set to deliver 750 tokens per second (!) running GPT-5.6 Sol, thanks to its wafer-scale compute allowing a model to run on a single piece of silicon. Google will effectively bake the design of Gemini into a future “Frozen” chip—mixing some metaphors, admittedly—while OpenAI has worked with Broadcom on its Jalapeno AI processor.
Helios is the realization of all AMD’s building blocks put together. Of course, it is one offering, likely with the kind of margin that only an “integrated solution” can command. But Dieckmann makes the important point that the most advanced data centres require such detailed coordination that even Helios-level systems engineering is too narrow. Enormous clusters of rack-scale systems require vast liquid loops, untold power delivery and space. Helios is a critical building block, but one of many.