Organizations are embracing AI en masse to accelerate processes. Without the right security, however, this innovation can quickly become a loose cannon. How do you ensure that the AI your organization uses runs securely and isn’t compromised? We’ll discuss this in an in-depth roundtable discussion with experts from ManageEngine, Nutanix, Okta, Thales, TrendAI, Veeam, and Zscaler.
For IT professionals and decision-makers, AI security can still be a bit unclear. Everyone is talking about it, but the concrete details remain abstract. To clarify exactly what it entails, we’ve brought together a diverse group of security experts across the tech stack. What does AI security really mean? Is it about full-stack security or rather specific runtime controls? And now that the market is realizing that AI comes with a hefty price tag, the question arises: how do we keep protection affordable and practical for the entire workforce?
It’s high time to take stock of the gap between the theory of the models and the stubborn reality on the ground. We’ll do that in a two-part series, of which this is the first installment.
Design flaw in the architecture
Securing AI applications is deeply intertwined with our fundamental approach to IT architecture and access management. Albert Kramer of Zscaler gets straight to the point on this issue, stating that AI poses a massive threat to what the industry has built up over the past decades. “What AI promises is the highly efficient use of data. But we’ve skipped a few crucial steps in granting that access,” Kramer explains. “In the world of zero trust, we actually start with no access at all. We decide what to grant access to based on the user’s identity or intent. Right now, we’re struggling enormously with that concept because we’re effectively giving AI agents unlimited access. This causes countless compliance issues and data breaches. It literally turns the past 25 to 30 years of cybersecurity on its head.”
Edwin Weijdema of Veeam agrees and notes that we’re making fundamental architectural errors. According to him, AI security is essentially what we’ve always done, but turned inside out. “We’ve bolted AI directly onto the data, thereby bypassing all the layers of defense we normally employ,” Weijdema explains. He draws a comparison with an amusement park: “It’s as if you have a ‘fast-track lane’ that lets you go straight into everything. We’ve always built security around human identities. We kept out whatever we didn’t know. Now, the unknown suddenly has direct access to our data. That’s why we need to introduce a completely new ‘trust layer.’ AI security encompasses everything that touches the data as it passes through the AI model’s pipeline.”
Proactive versus reactive
As soon as AI is truly integrated deep into business processes, the vulnerabilities change as well. Cybersecurity used to be largely reactive, focusing on plugging leaks after they were discovered. AI, however, requires a proactive approach. Bart Herps of TrendAI warns: “Everything now revolves around speed. AI still relies on traditional IT: the network, storage, and hardware. The difference is that if there’s a configuration error or vulnerability now, it’s exploited at an extremely rapid pace. A data breach involving AI escalates at lightning speed. We must therefore shift from a reactive to a proactive security mindset.”
Praveen Das of ManageEngine adds that the nature of the cyberattacks themselves is also changing. The threat landscape is no longer what it used to be. “With a traditional vulnerability, you have a pattern or a signature that you can patch against. But here, ‘jailbreaking’ is one of the biggest problems, and that doesn’t carry any signature at all,” Das emphasizes. He argues that security must be holistic. “You can’t secure just one part of your AI. It has to happen at all levels—in your infrastructure, model, and data.” Only then, in his view, can you be truly effective. “Think of data poisoning, which alters your model’s output, or an agent that suddenly tries to access documents it’s not authorized to view.”
Hundreds of invisible AI agents
One of the most complex challenges in today’s landscape is the popularity of agentic AI. These autonomous agents perform tasks on behalf of the user. Whereas traditional security focuses on human employees, these agents operate completely autonomously. And there are many of them. Weijdema notes that the balance has been completely skewed: “We’re already seeing scenarios where 82 agents are running for a single physical person. These agents appear, do their work, and are sometimes gone again five minutes or five seconds later. How do you build a system that can handle that?”
Rob Sanders of Okta delves deeper into this identity crisis. “We need to look at who is performing the action: is it the agent or the human user in whose context the agent is running? You’re dealing with multiple identities.” Sanders points out the need for fine-grained authorization. “If, as an employee, I have full write access in Salesforce, that doesn’t mean the AI agent I’ve built should have the same rights. We see agents that launch ten additional agents themselves to perform a task and then shut them down again. It’s impossible to secure that with traditional tools. We need to treat these agents as full-fledged identities, use short-lived tokens, and ensure we can detect them.”
Stephan Wibier of Nutanix adds another important dimension to this: the underlying intent. “Is it an agent that has gone ‘rogue,’ is it simply a very efficient employee, or is it an attacker trying to use targeted prompts to make the LLM dump its entire context? How do we make that distinction?” Wibier wonders. According to him, this clearly demonstrates that security must be seamlessly integrated into the platform from day one and cannot be tacked on afterward.
A new nightmare for the CISO?
The drive for workplace efficiency means employees aren’t waiting for the IT department to build the perfect, secure AI solution. They’re searching for the most convenient tools on the internet themselves. This is leading to an explosion of Shadow AI. Bart Herps of TrendAI shares a telling real-world example: “I recently visited a client who saw in their monitoring tools that more than 250 different cloud-based AI applications are currently being used within their network. And that number is growing every week. However, they can’t simply block these tools, because they’re sometimes seamlessly integrated into critical production processes without anyone noticing. For example, they use smart glasses that perform specific tasks with AI, but the IT department doesn’t know exactly which AI is running behind the scenes.”
To the CISO, this sounds like pure chaos. Herps aptly describes the dynamics in the boardroom: “When you talk to senior management, they talk about productivity. When you talk to the IT manager, he sees complexity. But when you talk to the CISO, he sees panic and chaos.”
Rob Sanders of Okta understands that initial panic reaction well. “If, as a CISO, you’re personally responsible for data breaches, your first reflex is to shut everything down,” he admits. “But just like shadow IT, we’re not going to stop shadow AI. After all, it’s good for productivity. What we do need to do is build guardrails. You set up frameworks where employees can work with their own tools, but where the system simply blocks access as soon as an unauthorized model tries to access your sensitive company data.”
According to the rest of the panel, frantically blocking everything is therefore not the solution. Kramer warns that the genie is already out of the bottle. “Users see AI as a toolbox. The corporate tool might do 80 percent of what they want, but for that remaining 20 percent, they’ll just as easily upload their data to an unknown cloud application because it’s faster.” Weijdema, however, also sees an opportunity here with shadow innovation. “Employees use these tools for a reason. They want to accelerate the business. The CISO’s role must therefore shift from being the ‘no department’ to the department that ensures business continuity. Facilitate the need, but do so through controlled, approved channels.”
The blind spot in the pipeline
Steven Maas of Thales, in the discussion about the countless models and agents, points to the absolute foundation: the data. “It all starts with data,” Maas states resolutely. “No matter how advanced your controls are, if you have sensitive data, you must protect it at its core with encryption. You must prevent your data from being manipulated. Models, after all, find paths and connections that we previously didn’t think were possible.”
This risk is amplified when using RAG (Retrieval-Augmented Generation), a technique in which AI searches directly within a company’s own documents. Herps warns of the dangers of this: “Even if you create your own RAG, you must be absolutely certain that the data within it is secure. A RAG system breaks all your data into small pieces. It doesn’t understand the data itself—it only knows the pieces—and the language model then stitches them back together. You can secure traditional files with permissions, but you often can’t do that with the pieces in a RAG.”
According to Weijdema, many companies are also struggling with the “illusion of progress.” “We’ve moved to the cloud; we have AI and agents, but we’ve forgotten about the data layer. We have 30-year-old data on our servers that nobody uses anymore, but is full of PII. If we don’t clean that up first, we’re feeding bad and sensitive data into our models, and chaos will ensue at lightning speed. AI is now forcing us to retroactively get our data governance in order.”
Proprietary models, firewalls, and control over workloads
Because public LLMs pose risks, some organizations are taking matters into their own hands. Das explains how his company made this decision: “We have deliberately eliminated the need for third-party LLMs by developing our own ‘narrow LLMs.’ In terms of pure computing power, these may not be comparable to the giants, but they’ve been specifically trained for our IT use cases. Because we build our own models and RAG, we have full control over permissions and ensure that no biased data ends up in our training model.”
Moreover, building our own solutions is sometimes the only option from a sovereignty perspective. Maas offers a word of caution about trusting tech giants blindly. “How can I trust someone else’s model? You wouldn’t be the first to discover that there are elements in an external model that don’t belong there. Let’s not be naive about the practices of certain large cloud providers.” For that reason, Thales builds in specific safeguards, Maas explains: “For customers in sensitive sectors such as defense or finance, we’ve created AI firewalls that prevent certain data from being used, uploaded, or even leaving the organization at all.”
Ultimately, it all comes down to understanding the workload—the actual task being performed. Das compares it to a restaurant: “It doesn’t matter where the chef was trained; what matters is what’s on the plate tonight. That order slip contains the audit: which dish is being cooked, and which allergies need to be taken into account? The workload must be more secure than the model itself.”
The end of the network
As the discussion progresses, it becomes clear that the traditional concept of the castle wall has definitively been rendered obsolete. Wibier emphasizes that a large part of AI security is actually just robust infrastructure security. “Seventy to eighty percent of the security that AI needs should already be in your stack. Think of network security. If you don’t have visibility, and a hacker performs a network injection, then you’re in trouble.”
Kramer adds that the concept of “the network” has fundamentally changed. “We’re moving away from a world where everything is on-premises. What is the network these days? It’s the internet. You simply can’t protect the network anymore.” According to him, the solution lies in drastically reducing the attack surface. “You have to protect the workload. That means you have to micro-segment everything: the dataset, the user, the agent. Based on that micro-segmentation, you determine for each specific action whether it’s allowed or not. And then it no longer matters whether the action takes place at the company’s office, at someone’s home, or from an internet café. As long as you operate according to the philosophy: if it can’t be reached, it can’t be hacked.”
From blocking to foundations
The successful and secure deployment of artificial intelligence ultimately requires much more than simply shielding an algorithm with an extra tool. It demands a thoughtful and proactive overhaul of the entire IT architecture. From cleaning up and encrypting raw, thirty-year-old data to dynamically assigning identities to ephemeral AI agents: everything must fit together seamlessly to prevent catastrophic data breaches.
In addition, the roundtable discussion shows that frantically blocking everything is a dead-end approach. Bridging the gap between the drive for innovation on the shop floor and the IT department’s frameworks is essential. By acknowledging this new reality and building on a modern, updated vision of zero trust and microsegmentation, cybersecurity can keep pace with the speed of AI. Only then will security transform from a barrier into the absolute foundation for sustainable business continuity.
This was the first part of a two-part series on AI security. In the next article, we’ll delve deeper into the solutions, governance, required skills, and cost implications of AI security.