Red Hat AI 3.5 centers on pre-deployment security verification, multi-tenancy on shared GPU infrastructure, and observability. Agent-based development also gets a boost with AutoRAG and ready-to-use agent templates.
Red Hat AI 3.5 is designed to provide organizations with the operational foundation to run AI workloads with the same discipline as other business-critical infrastructure. This includes provable security, cost allocation, and tight resource management.
This is achieved, in part, through the general availability of EvalHub. This tool enables teams to pre-screen models, RAG configurations, and agents for risks and link them to auditable compliance reports. Over twenty validated models have been added to the model catalog, including Google’s Gemma 4, Nvidia’s Nemotron 3, and Alibaba Cloud’s Qwen. In addition to performance benchmarks, these models now also receive Garak security scores and risk scores for PII exposure and toxicity.
Multiple tenants on the same GPUs
The second focus is on shared GPU infrastructure. Fair-share scheduling distributes capacity across tenants, while priority-aware serving protects real-time inference and lets background tasks use the remaining capacity. For those who need stricter separation, Red Hat AI now officially supports hosted control planes on OpenShift Virtualization. Each tenant then receives its own control plane, with VM isolation on top of shared hardware.
The distribution is also expanding. According to the release notes, the scheduler for distributed inference is getting new scorers that improve KV cache reuse. LLM-D is expanding beyond OpenShift: it’s generally available on CoreWeave CKS and Microsoft Azure, with Amazon EKS as a technology preview.
Agents with guidelines
For agentic applications, Red Hat is introducing AutoRAG, which connects enterprise data sources directly to agent applications, including multilingual document support and contextual retrieval. The Responses API with built-in RAG is becoming generally available, enhanced with NeMo Guardrails that intercept malicious tool calls. Agent templates for use cases such as code review, document processing, and research workflows are now available in AI Hub. Inference-Time Scaling adjusts computing power based on query complexity.
On the observability side, dashboards are being introduced for inference health, GPU utilization, and model performance, plus per-user token metering for showback. MLflow provides visual agentic tracing.
Red Hat AI 3.5 is generally available immediately.