Server chips are diversifying based on customer needs, be it raw performance or high efficiency. That’s Intel’s proposition at the launch of Xeon 6, which is otherwise very conservative despite all the AI hype. Meanwhile, AMD is promising performance that puts Epyc in the lead, with two variants as well.
Around Computex, both AMD and Intel have opted to unveil their latest chip offerings. Not only for consumers with Ryzen 9000 and Lunar Lake, respectively, but also in the server area. We will first discuss the innovations regarding Xeon 6, which has just been announced. Earlier, AMD revealed the fifth generation Epyc, which we will look at next.
Xeon cores in 2 sizes
For the first time, a new Xeon series appears with different types of cores. Formerly known as ‘Granite Rapids’, the power-oriented Xeon 6 is a straight replacement for Xeon 5. Up to 128 cores are present, a doubling from the previous Xeon generation. These include Redwood Cove cores, including support for AMX FP16 acceleration, needed for many conventional AI workloads.
This attention to AI highlights Xeon’s suitability as a highly modern option for data centers. However, the E-core variant of Xeon 6 is a lot more striking. Previously, this product was known by the codename Sierra Forest, which was already announced to use Crestmont cores. This offering is not designed for AI specifically, but intended for lots of large-scale bite-sized workloads. Intel itself cites microservices as an ideal use case. A staggering 288 of E-cores are on board at most with the efficient version of Xeon 6.
Intel chooses awkwardly unwieldy names in its presentations to distinguish the Xeon 6 options, but from here on out, we will refer to them as Xeon 6-P for the version with P-cores and Xeon 6-E for the E-cores option.
Meteor Lake XL?
Observant readers know that these core architectures are familiar territory. This is in contrast to the just announced Lunar Lake for (niche) laptops, which will appear in Q3 2024. Xeon 6, in both manifestations, is simply a repeat of the architecture behind Meteor Lake, only found in laptops. The difference with Xeon, apart from its focus on data centers, is that each chip is homogeneous. There’s no mix of P- and E-cores here.
It is not surprising, as Xeon 6 has been discussed in detail at length under its previous code name. Still, the official unveiling of Xeon 6 as such is a good time to bring up a strategic choice by Intel. Where Nvidia prides itself on debuting its own GPU architectures within data centers, Intel’s CPUs are conservative. More than three-quarters of all server racks run Xeons, so some conservatism is perhaps to be expected.
Less rack space, less power
As usual, Intel mainly contrasts its latest Xeon with the generation before it as well as with products from five years earlier. The latter is done mainly because a significant portion of data center users deploy this period to upgrade their own servers. With this 5 year refresh cycle there is an important choice awaiting Xeon 6-P or Xeon 6-E. It seems that for these customers, Intel recommends the E variant most clearly, possibly because users with such a slow upgrade cycle do not seek the heaviest CPU workloads. More often than not, the improvement here is just over 2.5 times (web, microservices, media, data services), with an outlier in networking (3.4 times faster).
Xeon 6-P is often compared to Xeon 5, highlighting AI (2x), High-Performance Compute (HPC, 2.3x) and general purpose computing (2x). AI inferencing stands in stark contrast to AMD’s earlier Epyc offering with a 3.7x boost, although that company now promises big performance gains in that area as well.
The 6700E version of Intel Xeon, equipped with E-cores, appears today. The 6900P (that is, a P-core variant) will follow in Q3 of this year, while other options will appear in Q1 2025. As for the 6700E, Intel stresses that customers can free up a huge amount of sever racks for the same tasks as before. Where 200 racks are needed now with a 2nd-generation Xeon from five years ago, 66 will suffice with the 6700E. That will save 80,000 MWh over a four-year period and 34,000 metric tons of CO2, according to Intel.
AMD continues to grow with Epyc Turin
While Intel has been offering details on Xeon 6 for some time, AMD is bringing a lot of Epyc details at Computex for the first time. It has made the same choice as Intel when it comes to multiple cores: there is both an Epyc line with full Zen 5 cores and one with Zen 5c. Note that with AMD, the c versions of the cores are a lot closer to the regular option. Epyc includes up to 128 Zen 5-cores or 192 Zen 5c-cores.
It results in a 5.4x performance gain in terms of AI inferencing over Xeon 5. So those running a chatbot on the CPU are many times better off with an Epyc chip currently. It looks like that conclusion will remain the same with the upcoming generations, although exact performance won’t be clear until after launch. Both Intel and AMD can only compare “fairly” with the competitor’s somewhat outdated product, since logically they do not give each other their own engineering samples.
Interestingly, Epyc now owns a third of the server market, according to AMD CEO Lisa Su. It is catching up with Xeon, which owned 80 percent of this sector in 2021. AMD has grown from 11 percent to 33 percent since that year. We see no reason to expect a trend reversal in this area, especially since parties like OVHcloud are increasingly offering AMD options for their servers. This is exactly the adoption AMD needs and is not receiving from OEMs on laptops for now. Now that that is happening on servers, Intel’s dominance is breaking down.
AMD is additionally innovating with options like Epyc 4004, which delivers server functionality on a desktop socket. Since the death of Intel’s workstation Xeons a few years ago, there has been no equivalent of these in that company’s offerings. Combine that with Microsoft’s recent announcement that it is making AMD AI chips available on Azure as an Nvidia alternative, and it becomes clear that AMD is well on its way.
Read more: Microsoft offers AMD’s AI chip as alternative to Nvidia
Wattage skyrockets
We do question some of the stated consumption with both AMD and Intel. Both Xeon 6 and fifth-generation Epyc are going for up to 500 Watts for the top models. That not only requires a lot of power, something much more costly in the long run than the initial purchase price, but forces server farms to adopt new cooling solutions.
However, customers do get cheaper power consumption if workloads remain the same compared to the old server chips. For existing fourth-generation Epyc customers, this is an attractive option, since socket retention allows a drop-in upgrade. With Xeon 6, this is not the case.
Also read: Nvidia competitors boost AI hardware development for data centres