As containerized applications are increasingly deployed by enterprises, more and more technologies are emerging to more effectively organize your Kubernetes environment. For example, storage vendor DataCore has been offering Bolt for a few months now, which arranges storage to run stateful applications within containers. We talked about the combination of Kubernetes, containers and storage with Senior Solutions Architect Pieter van de Burg.
The fact that DataCore recently launched a dedicated Kubernetes product is not surprising in itself. Basically, with Software-Defined Storage, the company wants to place a software layer over the storage hardware, for more efficient and smarter design and management of storage. Until recently, it had the three core products SANsymphony, vFilO and Swarm for this purpose. These three products address Block, File and Object storage respectively.
Also read our story about DataCore’s overall strategy: DataCore Software-Defined Storage simplifies and improves data access
However, the market is increasingly moving toward containerized applications. Kubernetes is often chosen for automating and managing cloud-native software. However, modern applications using Kubernetes also require custom-built storage services. That’s why DataCore recently introduced Bolt, meaning it now has four core products.
Road to Kubernetes and container storage
The launch of version 1.0 of Bolt did not come out of the blue, Van de Burg makes clear to us at the start of our conversation. Kubernetes is a platform that has been around since 2014, after which it was rapidly adopted within the cloud-native world. DataCore already saw in 2017 that we were at the beginning of the container revolution. At that time, they saw a particular role for containers in combination with the SANsymphony product. This is because SANsymphony focuses on block storage, where high availability and performance are very important. For software that uses containers, these characteristics are also a priority.
DataCore therefore set up Project CDS (Container Data Services) internally. With this project, the company made internal resources available to develop persistent storage for containers on the basis of SANsymphony. In doing so, the company opened up the SANsymphony data services to Kubernetes. To distribute persistent storage to applications, a CSI (Container Storage Interface) is required. This plugin for Kubernetes and other container orchestrators allows storage providers to serve persistent storage to containerized applications. The SANsymphony CSI plugin focuses on legacy environments, basically those infrastructures that are not cloud-native and are not moving to the cloud. Within these types of environments, we are increasingly seeing the need to start using containers in addition to legacy applications.
Within legacy environments, DataCore can provide high-performance and highly available persistent storage for containerized applications.
Text continues after the box
Use case
In this article, we talk mainly about Kubernetes storage. We asked DataCore if it could also share a real-world example, to make it a little more tangible. It came up with Civo, a service provider managed entirely from Kubernetes. Earlier this year, Civo shared what it did to make Kubernetes a go-to choice for large enterprise organizations.
Civo saw that changing the default storage layer for clusters was crucial to improving reliability. The service provider has completed the move, making new clusters use OpenEBS Mayastor as the default storage layer. Civo is also in the process of migrating all existing clusters to Mayastor. “We’re really excited to complete this since our testing has proven that it improves stability to the platform and is also superior in performance.”
Achieving true cloud-native
Although the focus on Container Storage Interface was a first step to Kubernetes for DataCore, the company also saw that more was needed. Van de Burg says that DataCore actually had to free up too many resources internally. So, in early 2020, the company decided to invest in MayaData, transferring the people who were working on containers for DataCore to MayaData as well. MayaData is the main developer and contributor within the open-source storage technology OpenEBS. Intellectual property and knowledge were also brought to MayaData. From there, both parties continued to build, from which came the OpenEBS Mayastor engine.
OpenEBS Mayastor allows developers to more easily deploy Kubernetes stateful workloads. Basically, it does this by managing available storage on each Kubernetes node and using the storage for Local Volumes (for distributed workloads like MongoDB and Cassandra) or Distributed Volumes (for stateful workloads like Percona, MySQL and Jira). Advantages of OpenEBS Mayastor are that container storage is custom-built, which offers Kubernetes apps agility and flexibility. Also, containerized software can be run on any cloud.
For ITOps and DevOps
Van de Burg sees that SANsymphony and OpenEBS with the CSI plugin are two products with very different goals. The CSI plugin for SANsymphony was written to start using containers from legacy architectures. The plugin is, as it were, a first step towards containers and, according to Van de Burg, is especially usefull for ITOps.
OpenEBS Mayastor, in turn, is the real cloud-native story. This often appeals to the DevOps world, Van de Burg argues, since they are coming from the cloud. Both in the development environment and in production, DevOps prefers Kubernetes. For software then, performance and availability are often important features. In that case, developers can turn to OpenEBS Mayastor.
Storage controller Bolt addresses speed, performance and scalability
Ultimately, it was the OpenEBS Mayastor engine open-source technology that led to the introduction of Bolt. Here’s the thing: in late 2021, DataCore acquired MayaData, promising to keep the technology open source while using the company’s expertise to develop its own technology. Bolt thus became the commercial Kubernetes storage product based on MayaData’s technology. The reason for DataCore to come up with a dedicated container storage product was that developers cannot build modern software and microservice architectures on traditional infrastructure. Storage actually needs to be rebuilt from the ground up, to eliminate manual tasks and harness the potential of containers.
When we ask Van de Burg about the uniqueness of the new Bolt product, he particularly praises the high performance DataCore achieves for stateful workloads. “We can actually deliver the fastest storage towards the containerized applications,” says the Senior Solutions Architect about the position in the market and competition with other vendors. Bolt achieves the high performance by leveraging the capabilities offered by NVMe and NVMe over Fabric. That is, the software solution uses the NVMe protocol across the entire data path. In this way you obtain minimal overhead and the latency advantages of NVMe can be fully exploited.
Van de Burg sees DataCore, compared to competitors, moving forward for legacy environments with SANsymphony’s CSI plugin. Bolt is deployable for cloud-native apps/microservices. Bolt technology can also deliver high-performance, highly available storage for containerized software based on standard hardware in on-premises environments.
DataCore draws great confidence from the fact that with Bolt it can now actually offer Software-Defined Storage for any situation: Containers, Block, File and Object Storage.
Ultimately, Bolt fits the bill for DataCore services because of its strong focus on providing a variety of data services. In the case of Bolt, you can think of streamlining the Continuous Integration and Continuous Delivery pipelines, using storage services orchestrated directly from Kubernetes. There are also services for removing complex storage management (thanks to Kubernetes automation) and visibility into storage behavior.
Deployment and scalability
To make use of Bolt, DataCore offers the product as a software package. This package can be deployed on supported nodes within a Kubernetes cluster. These can be bare metal, virtual or cloud-hosted nodes. Here, DataCore emphasizes being an independent software vendor. For Bolt, it means supporting any NVMe-powered storage hardware. When Bolt is on a node, it creates a storage pool for Kubernetes containers. Big advantage of the software-based approach is also that it makes storage enormously scalable.
Supporting different situations
Bolt is the product that DataCore has been working toward for about five years. With it, it can deliver the storage controller for Kubernetes with which it is aiming for substantial growth. It is expected that specialized storage products will have a bright future due to the increasingly prominent role of containers within enterprise environments,
At the same time, DataCore continues to have its eye on the CSI plugin for SANsymphony and OpenEBS Mayastor. The CSI plugin will be more prominent in legacy environments, while the OpenEBS Mayastor remains an open-source project. With that, Datacore has really set its sights on the Kubernetes future.
16 minutes ago
