F5 has opened its AI Gateway to a limited number of customers in early access. This containerized toolset is designed to streamline interactions between LLMs, the APIs that access them, and the apps that use them. AI Gateway also adds a dose of security. The goal is to keep enterprise organizations adopting AI.
A recent report from F5 confirms that ever more companies are integrating AI into their work processes. To be exact, 75 percent of the organizations surveyed are doing so. They will eventually look for ways to run AI workloads faster, more efficiently, more cost-effectively, and more securely. F5 expects to capitalize on that with its AI Gateway.
The new product promises improved data quality, observability and threat protection by monitoring GPU cost management, as well as managing system responsiveness and compliance requirements. It integrates with F5’s existing portfolio of solutions for optimizing, securing and scaling (cloud-native) applications.
Key threats neutralized
For example, The F5 AI Gateway provides automated compliance checks against the ten key threats defined by OWASP, the Open Web Application Security Project. This ensures that applications built with LLMs comply with critical security standards. This feature simplifies addressing vulnerabilities and protects against the most common threats, such as injection attacks, broken authentication and insecure data exposure.
Another key feature is semantic caching, which intelligently manages repeated or similar commands by storing them for later use. Because the response to a new but similar task does not have to be created from scratch, this feature reduces the workload for the LLM on duty. This ensures faster response times and less computing power required.
Reliably running heavy loads
In addition, AI Gateway comes with streamlined API integrations that simplify the often complex connections required to deploy AI models effectively. These integrations allow developers to focus on building advanced functionality (where their business makes its money, presumably) rather than busying themselves with the intricacies of the underlying infrastructure.
Finally, load balancing and rate limiting ensure the reliable running of heavy loads. Meaning services stay up and running, delays are minimized, and the customer generally keeps a stable and efficient AI ecosystem running.
Meeting a need
“LLMs are unlocking new levels of productivity and enhanced user experiences for customers, but they also require oversight, deep inspection at inference-time, and defense against new types of threats,” said Kunal Anand, Chief Innovation Officer of F5. By adding this new gateway to F5’s existing API traffic management tools, the company hopes to meet exactly these needs.
The AI Gateway works in both cloud and data center environments (as well as edge networks) and is compatible with F5’s NGINX and BIG-IP platforms (a web server/load balancer and application delivery platform, respectively). Its scalability allows customers to adapt their security policies on-the-fly. Meaning the product should be able to deal with future AI activities and the additional compliance requirements that these come with.
Also read: Four stages to observability: up to 2.6 times more return on investment