Software system identity security company CyberArk wants to bring clarity to the world of artificial intelligence model use with the release of its FuzzyAI open source framework. The company says that its technology has “jailbroken” every major tested AI model in order to assess, audit and quantify its fragilities and frailties.
As data scientists will know, jailbreaking is a software process designed to manipulate large language models (LLMs) into a state where they produce inappropriate (or even harmful) content that can sometimes be used for malicious purposes. FuzzyAI helps organizations identify and address AI model vulnerabilities such as “guardrail bypassing” (that would enable the model to act outside of its defined scope and parameter) and lead to harmful output generation.
FuzzyAI works in cloud-hosted and on-premises in-house AI models.
Fuzzy clarity
According to Peretz Regev, chief product officer at CyberArk, I models are transforming industries with application use cases that span customer interactions, internal process improvements and automation.
“Internal usage of these models also presents new security challenges for which most organizations are unprepared,” said Regev and team. “FuzzyAI helps solve some of these challenges by offering organizations a systematic approach to testing AI models against various adversarial inputs, uncovering potential weak points in their security systems and making AI development and deployment safer.”
Finger on the fuzzer
At the heart of FuzzyAI is a powerful “fuzzer” i.e. a software a tool that reveals software defects and vulnerabilities and is capable of exposing vulnerabilities found via more than ten distinct attack techniques, from bypassing ethical filters to exposing hidden system prompts.
“The launch of FuzzyAI underlines CyberArk’s commitment to AI security and helps organizations take a significant step forward in addressing the security issues inherent in the evolving landscape of AI model usage,” said Regev. “Developed by CyberArk Labs, FuzzyAI has demonstrated the ability to jailbreak every major tested AI model. FuzzyAI empowers organizations and researchers to identify weaknesses and actively fortify their AI systems against emerging threats.”
Key features of FuzzyAI include so-called “comprehensive fuzzing” where the tool probes AI models with various attack techniques to expose vulnerabilities like weak guardrail controls, information leakage, prompt injection or harmful output generation. There is also a degree of extensible framework provisioning here so users researchers can add their own attack methods to tailor tests for domain-specific vulnerabilities.
In terms of fuzzer fuzzing techniques on offer here (yes, we did just say fuzzer fuzzing) the FuzzyAI fuzzer supports a variety of approaches mutation-based fuzzing, generation-based fuzzing and intelligent fuzzing. For input generation, it provides built-in input generation capabilities to generate valid and invalid inputs for testing. In terms of integration, CyberArk says that the FuzzyAI fuzzer can be integrated into existing development and testing workflows.
Community spirit
CyberArk insists it has a growing level of community involvement in this technology (it is open sourced, so we would hope for nothing less) and says that its community-driven ecosystem ensures continuous adversarial techniques and defence mechanisms advancements. The CyberArk FuzzyAI fuzzer is released and made available to developers and data scientists via the Apache license.