JFrog introduces Shadow AI Detection to help companies identify and manage uncontrolled AI use in software development. The feature is designed to help organizations balance innovation with security and compliance.
The rapid adoption of AI in development teams creates a governance challenge. Developers and data scientists regularly integrate AI models and services from providers such as OpenAI, Anthropic, and Google without central approval. This phenomenon, known as Shadow AI, introduces blind spots in the security infrastructure.
JFrog’s Shadow AI Detection is designed to detect these invisible AI assets automatically. The tool inventories internal models and external API gateways that provide access to data from approved or ad hoc sources. Yuval Fernbach, VP and CTO ML at JFrog, explains: “Recognizing and mitigating the risks of shadow AI is becoming a critical priority for CIOs and CISOs who must strike a balance between innovating while maintaining security.”
The software company is presenting the functionality at JFrog swampUP Europe. The solution is part of the JFrog Software Supply Chain Platform.
Governance and audit trails for compliance
Once detected, organizations can centrally manage AI assets. Teams can enforce security and compliance policies for all AI resources. They can define paths for authorized users to access third-party AI services.
In addition, the system monitors the use of external AI models and APIs, such as those from OpenAI or Gemini. This is becoming relevant due to emerging regulations. Fernbach emphasizes that organizations must follow proven software development practices with developer-friendly workflows, strong security, and robust governance.
Various legal and regulatory frameworks require complete audit trails of AI activity. The US Transparency in Frontier AI Act, the EU Cyber Resilience Act, the EU AI Act, Germany’s BSI Guidelines, NIS2, and the Guidelines for Securing AI Systems all set requirements for AI accountability.
Availability and implementation
Shadow AI Detection is available as part of JFrog AI Catalog. General availability is expected later this year. The company positions the functionality as an extension of its 360-degree approach to securing the AI supply chain.
The new detection capabilities help organizations ensure provenance, accountability, and resilience. They are designed to support responsible AI development by enforcing rigorous risk management and reporting standards. They also require visibility into software components and the securing of AI systems from design to deployment.