Databricks is introducing several tools to help organizations scale and deploy AI agents. The new functionality should enable organizations to deploy AI agents to mission-critical applications with appropriate governance and accuracy.
The extension to Mosaic AI Gateway enables centralized management of both open source and commercial AI models. This allows companies to add their own LLM endpoints and maintain full control over their specific capabilities while leveraging the Databricks platform.
The new Genie Conversation API suite allows developers to integrate natural language-based chatbots into in-house applications or popular tools such as Microsoft Teams and Slack. Users can run prompts and then get data-based insights as usual in the Genie UI. They can also ask follow-up questions.
Improve agent evaluation
Databricks also includes updates to the Agent Evaluation Review App to streamline feedback from domain experts. The tool allows users to collect structured feedback and systematically improve the performance of AI agents.
For organizations looking to scale AI agents, Databricks is introducing support for batch inference with AI Functions. This reduces infrastructure complexity. Users can perform batch inference through a single SQL query without having to set up infrastructure.