Without data, there is no AI. Yet data and storage have long been somewhat neglected in AI, as compute, and to a lesser extent, networking, received more attention. That is changing. At Supercomputing this week, Hammerspace is unveiling its own Hammerspace Data System, a single data system for AI. A key component of this is Hammerspace OFP, DPU-based storage that further simplifies the architecture.
We’ve been following Hammerspace for several years now. Until now, it was a software company and, in short, focused primarily on making all of an organization’s data centrally available via a global namespace, orchestrating where the data ends up, and accelerating it.
The latter is also known as Tier 0. A few years ago, Hammerspace developed a way to leverage the fast NVMe flash in GPU servers, storage that Nvidia itself never actually uses but which is often simply built into the GPU servers, for tasks that require the highest performance and, above all, the lowest latency.
From data platform to data system
Until now, the company’s offering was known as the Hammerspace Data Platform. Starting today, it will be called the Hammerspace Data System. It not only adds the previously mentioned Hammerspace Open Flash Platform (OFP) to the lineup but also introduces Data Intelligence and a new Tier 0 solution.
Today’s announcements also mean that Hammerspace is no longer merely a software vendor. It now develops storage hardware as well and, together with several partners, is entering the market for complete systems.
What is the Hammerspace Open Flash Platform?
About a year and a half ago, Hammerspace was one of the first members of the Open Flash Platform initiative. Others included the Linux community, Los Alamos National Laboratory, ScaleFlux, SK Hynix, and Xsight Labs.
According to the initiative’s founders, the existing model for connecting GPUs to data was, and remains, insufficient in the world of AI. It needs to be more efficient, faster, and less complex. The result is a DPU-based storage layer that is fully integrated. In other words, an x86-based server is no longer needed, and therefore, neither is a CPU. The GPUs can communicate directly with the storage, where the DPUs are integrated.

The DPUs contain all the necessary logic. In practice, that’s pNFS logic, since that’s the protocol used here. Beyond that, all you need is a Linux client (and, of course, GPUs) to get started. The fact that it’s pNFS means metadata and data can be separated, and performance scales very well.
Small, powerful, scalable
In other words, Hammerspace has succeeded in giving AI storage an incredibly small footprint. The DPUs don’t take up much space, and neither do the NVMe drives inside the appliance. In total, each rack unit (1U) can hold 40 SSDs, spread across five sleds.
Each sled has a DPU (from Xsight Labs) and two 400GbE ports. In total, a single unit can hold nearly 10 PB of raw capacity. Note, however, that this requires the use of QLC. To achieve the maximum read speed of 400 GB/s, TLC must be used instead.
It will be interesting to see how Hammerspace OFP, as a pNFS product, compares to how NetApp has implemented pNFS in the recently announced NetApp Novus. These are two different approaches to solving essentially the same problem: ensuring that data and storage are no longer the bottleneck for GPU systems. It’s not only interesting to look at performance, footprint, and the like, but also at how easily it can be integrated into an existing environment. Something new like OFP may not go off without a hitch at first.

Data comes to compute, not the other way around
In practice, this could mean simply placing one or two of these new units at the top of the rack, and then filling the rest of the racks with GPUs. If everything works as Hammerspace says it does, this is an extremely efficient way to add storage to compute.
What Hammerspace is doing here does go a bit against the grain. Many vendors talk about bringing compute to the data. With OFP, Hammerspace is literally doing the opposite. It brings storage and compute together, as it were, into a single system. In any case, it’s a system that can be purchased and deployed as a complete package. For the latter, Hammerspace has already lined up several partners: Cisco, Hitachi Vantara, and Supermicro.
Data intelligence
Besides the major announcement regarding Hammerspace OFP, we’d almost forget that the company has another quite significant announcement today. With Data Intelligence, Hammerspace is adding something very important to the complete system it intends to offer with its own Data System: insight into data, the ability to interact with that data, and take action when, for example, a policy needs to be set.
As noted, Data Intelligence is undoubtedly important. However, Hammerspace isn’t the first to introduce this. Earlier this year, we wrote about how Everpure announced a feature that’s not only also known as Data Intelligence but also does essentially the same thing.
That’s not to say Hammerspace’s announcement has no value. Quite the contrary. Without Data Intelligence, you can’t claim to have a Data System, so it’s an excellent move by the company. It’s just important to recognize that it isn’t hugely revolutionary, at least not as innovative as Hammerspace OFP, even though both are part of the bigger picture.