IBM and Groq are joining forces to bring agentic AI to businesses faster. The parties promise five times faster AI inference than traditional GPUs by combining IBM’s watsonx Orchestrate with Groq’s specialized hardware. Strictly regulated sectors such as healthcare will be able to benefit most from the new possibilities.
IBM is opening the door to GroqCloud via watsonx Orchestrate. IBM will specifically run the Granite models on GroqCloud. The combination of these two provides an infrastructure that allows companies to scale up at a faster pace.
The parties report that Groq’s hardware delivers “more than five times faster and more cost-efficient inference than traditional GPU systems.” In this way, the company is competing in Nvidia’s playing field.
Faster deployment of AI agents
At the heart of the collaboration is watsonx Orchestrate, an IBM solution that provides an overview of all AI agents, workflows, and enterprise tools present in a company. This tool can deploy both pre-built and customized agents for specific business tasks.
The integration with Groq’s infrastructure should help IBM customers bring their AI agents into production faster and more cost-effectively. According to Rob Thomas, senior vice president of software at IBM, large organizations have many options for AI experimentation, but it becomes complex when they want to move to production environments.
IBM emphasizes that the combination of speed and orchestration is particularly important for highly regulated sectors such as healthcare and financial services. Groq has a compliant platform that meets the most stringent regulatory and security requirements. In addition, these sectors can benefit greatly from the speed of Groq hardware. The combination gives IBM AI agents the ability to analyze data in real time.
Further announcements
The companies also announced expanded support for virtual large language models and Red Hat’s llm-d framework for large-scale, distributed inference on Groq’s architecture.