Today Intel launches the first Xeon 6 processor for data centers and the long-awaited Gaudi 3 AI chip. Xeon 6P is focused on pure performance, while the argument for Gaudi 3 is mainly based on lower cost and broad compatibility.
Last year, Nvidia CEO Jensen Huang claimed that organizations worldwide needed to start converting their data centers into “AI factories.” In other words, he was imploring clients to adopt infrastructures with a heavy focus on GPUs. Such an approach comes with an almost insatiable power appetite. Intel sees the world a little bit differently and focuses on continuity. This isn’t surprising, since Xeons still dominate the data center world despite a declining market share.
Xeon 6P: the powerful option
For the first time, Intel is building the Xeon line in multiple ways: with performance cores (p-cores) as well as efficiency cores (e-cores), but not both on the same product. The microarchitecture per core is the same as in Meteor Lake, the laptop chips from late last year for thin-and-lights. However, rather than “Meteor Lake XL,” there is a dichotomy. Indeed, Intel sees that some data center customers demand pure performance above all else, while others clamor for efficiency. To serve both camps, two products are needed.
First launches Xeon 6P, with up to 128 cores. Packaging, or the formation of all chip components, differs. The 128-core variant requires three compute tiles, while variants with single tiles can hold 16 or 48 cores. The intermediate form with two tiles stretches to 86 cores. Each Xeon 6P is built with the Intel 3 process for compute, with the much less complex I/O being on Intel 7.
In this, Intel is building on the Lego-like constructions that modern Intel and AMD chips have adopted. Each part is modular and packaging can be shuffled for more cache, memory directly on the chip and/or more economical chip manufacturing techniques for lower complexity parts. At AMD, the term “chiplets” is used, while Intelspeak refers to “tiles” and the exact implementation is different, but fundamentally the benefits and thought processes behind them are similar.
Xeon 6P is a CPU and by its nature widely applicable for all kinds of workloads. Still, HPC and the handling of large-scale data workloads are the most important tasks. In that area, Xeon 6 often beats Xeon 5 by 30 percent or more, as Intel shows from its own benchmarks. More important is the nearly two times better perf-per-Watt, meaning the same workload simply consumes half as much power on the new product.
Gaudi 3: a real challenger?
Gaudi 3 is finally launching, and it has had a long run-up. In April, we actually already knew an awful lot about the GPU that is intended to challenge Nvidia and AMD. As then, Nvidia’s H100, the AI chip to beat in the data center space, is the point of comparison. Although performance is higher in inferencing compared to the H100, the Intel story around Gaudi is not so much focused on pure performance. Instead, efficiency and a low total cost of ownership are paramount. Those running inferencing on Gaudi 3 and not the H100 will run them at a 1.8x better performance per watt, according to Intel.
Read more about Gaudi 3: Intel Gaudi 3: the most powerful AI chip for impatient Nvidia customers
The TDP is high at 600W, but lower than that of the H100 (700W). 128GB of HBM2e memory makes a single card deployable for smaller GenAI models.
Two servers with Intel Gaudi 3 chips on board will appear in October. Dell will launch the PowerEdge XE9680 with air cooling; Supermicro will offer both air and water cooling with the X14. For the latter, the Xeon 6 line will also take its place in the CPU socket.
These are big names, but it is by no means an avalanche of options. Compare it to the enormous list of vendors with Nvidia machines (with the H100 on board alone) and the contrast is extremely stark.
What Intel offers, however, is broad interoperability. Unlike Nvidia’s own standards for connectivity, Intel opts for the well-known Ethernet for intercommunication between AI chips. This is crucial for GenAI workloads that require multiple GPUs simultaneously. It also explains why Gaudi offers Gaudi 3 as an accelerator card, a universal baseboard with 8 Gaudi 3s and a conventional PCIe video card, providing maximum flexibility.
Also read: No, Qualcomm will not acquire Intel