Companies are eager to move AI models out of the experimental phase and into production environments. This comes with challenges such as increased hardware costs, privacy concerns, and a lack of confidence in sharing data with SaaS-based models. Red Hat sees the need for change in this area; otherwise, companies will fall behind in the rapidly changing AI world. Enter an AI platform that can run on-premises or in the cloud.
A typical company must implement IT modernisations to deploy AI successfully. Of course, what needs to be done will vary from organization to organization, but according to Red Hat, it often comes down to a few things. The obvious ones include modernizing existing application and data environments, removing barriers between existing systems and storage platforms, and deciding where workloads should run. What goes to the cloud, what to the data centre and what should run at the edge? Or will a combination be chosen?
To elaborate, Red Hat announced during the keynote “The cloud is hybrid. So is AI.” at Red Hat Summit in Denver updates. They aim to improve flexibility in the hybrid cloud, empower foundation models with enterprise data and enable a variety of hardware and software accelerators. This article discusses what Red Hat will be doing with OpenShift.
Red Hat OpenShift AI 2.9
The enhancement that stands out at the product level is the introduction of Red Hat OpenShift AI 2.9. Red Hat describes this solution as an open hybrid AI and machine learning platform built on Red Hat OpenShift. The latest version of this platform helps data scientists develop models via project workspaces and additional workbench images. They also get more options to use the development environments and toolkits they desire. Examples cited by Red Hat include Visual Studio Code and RStudio.
To support more locations, Red Hat OpenShift AI 2.9 comes with model serving at the edge. This extends model deployment to remote locations using single-node OpenShift. It provides inferencing capabilities in environments with limited resources or air-gapped network access. On top of that comes something Red Hat describes as enhanced model serving. It amounts to using multiple model servers to support predictive analytics and generative AI. Because of the extension, these types of analytics can be run on a single platform for multiple use cases.
Red Hat will also continue to support distributed workloads using the Ray framework. Ray uses multiple cluster nodes for faster and more efficient data processing and model training. The framework for accelerating AI workloads relies on KubeRay to manage workloads on Kubernetes. In addition, it uses CodeFlare, a framework within Red Hat OpenShift AI, to enable distributed workloads. Red Hat promises that the centralized management capabilities ensure that nodes are optimally utilized and support resource redistribution.
Also read: Red Hat ‘reimagines’ Enterprise Linux for AI
The hardware base
With the above features, there is a lot of innovation in the works to keep up as an enterprise in the cloud and AI era. However, Red Hat has also looked beyond its product by including partners in the story. Accordingly, during the keynote, the necessary new collaborations were announced with key industry players. What stands out is that in addition to Nvidia – a vendor present at every tech conference these days with a renewed partnership – the other major chip players, Intel and AMD, are also in attendance.
First, what Red Hat will do with Nvidia involves upcoming support for Nvidia NIM microservices on Red Hat OpenShift AI. Nvidia NIM is a set of inference microservices belonging to the chip company’s AI Enterprise software platform that allows organizations to accelerate the delivery of generative AI applications. The new partnership promises a streamlined integration path to deploying Nvidia NIM in a common workflow alongside other AI implementations. In addition, enterprises can count on integrated scaling and monitoring for Nvidia NIM deployments in coordination with other AI model deployments in hybrid cloud environments.
With the second chip manufacturer mentioned, Red Hat has partnered to facilitate the delivery of AI solutions on Intel AI products. The vendors will work together to ensure that Intel hardware is certified for Red Hat OpenShift AI to ensure interoperability and provide appropriate AI capabilities. This includes Intel Gaudi AI accelerators, Xeon processors, Core Ultra and Core processors and Intel Arc GPUs. All chips should be deployable for model development, training, management and monitoring.
Finally, AMD is another vendor Red Hat wants to work with more to support companies in hybrid AI/cloud scenarios. The AMD GPU Operators on Red Hat OpenShift AI will take a central role there, giving enterprises the processing power and performance for AI workloads in the hybrid cloud. According to Red Hat, this results in streamlining AI workflows and bridging existing and potential gaps in the GPU chain.
Story continues after box below
Red Hat OpenShift at Ortec Finance
Although this article mainly discusses Red Hat OpenShift AI, the original Red Hat OpenShift has been suitable for AI for a long time. The Dutch company Ortec Finance demonstrates that. This party provides software solutions for financial services companies. At Ortec Finance, they wanted to focus more on cloud-native applications. To this end, they built the Ortec Finance Cloud Application Platform, or ORCA for short.
Initially, Red Hat OpenShift formed the foundation for ORCA, but while testing the first version, it became clear that a fully managed platform was needed. This led to the implementation of Azure Red Hat OpenShift, which gave developers a wider choice of integrated and pre-validated tools. For Ortec Finance’s solution engineers and customers, it meant faster software deployment and managing applications on multiple Kubernetes clusters.
Using Azure Red Hat OpenShift, Ortec Finance can deliver higher-quality software faster. The option to release remotely has reduced turnaround times from days to minutes.
Ortec Finance appears to see advantages in applying more AI. For instance, over the past two years, it has been building machine learning models for home valuations on Azure Red Hat OpenShift to support institutional clients in efficient valuation processes. Currently, the Dutch company is implementing Red Hat OpenShift AI. With this, it aims to replace maintenance and proprietary code for delivering AI models in software. To do so, Red Hat OpenShift AI provides an environment that enables data collection and preparation, model training and fine-tuning, model delivery and monitoring.
Retrieval Augmented Generation
In the AI world, tech vendors cannot do without each other in this regard. That’s where many vendors have been finding each other lately. What all parties are also paying a lot of attention to in recent months when it comes to AI is improving the accuracy and reliability of models. Especially with generative AI, shortcomings appeared there. Retrieval Augmented Generation, or RAG for short, is seen as a solution. Through this technique, it is possible to add new information from which a model can draw, thus improving output. RAG makes it easier to integrate LLMs into business applications.
Also read our article explaining what RAG is and how it works.
Red Hat is also working to do more with RAG. During the Red Hat Summit, the company announced an enhanced partnership with Elastic. This will make Elasticsearch the preferred vector database for Red Hat OpenShift AI. This is particularly useful for developers using the Elasticsearch Relevance Engine, which features vector search and transformer models. This allows the developer to build search options incorporating enterprise data and bringing more reliable generative AI to software.
With the partnership around RAG, the new capabilities for Red Hat OpenShift AI, and the renewed collaborations with chip makers, Red Hat is showing that it is taking AI seriously. The open-source player wants to be exactly where companies are in their AI adoption. That way, every organization can harness AI’s potential.