Following the uproar surrounding the AI-driven Hugging Face hack and apocalyptic statements from departing employees at AI labs, OpenAI is changing course. In that hack this summer, OpenAI models escaped their sandbox during a security test via a zero-day vulnerability in Artifactory and then broke into Hugging Face’s infrastructure. The company is now sharing a framework for future disclosures of situations in which its own models go badly wrong. “Misalignment” takes many forms, as shown by the six alarming incidents that OpenAI is publishing alongside the framework.
The general theme of these six situations is that they reveal an underestimation on OpenAI’s part: of LLMs’ drive to answer a question correctly, of the unreliability of summaries that AI writes for itself, and of LLMs’ ability to wriggle their way out of a sandbox.
In other words, while it’s good to publicize these examples, they show that OpenAI was anything but prepared for the escalation in agentic capabilities that LLMs have in 2026. We also detect an approach based on “move fast and break things,” with fixes mostly aimed at specific problems. It almost seems like a guarantee that we’ll see more incidents. But what were these incidents about?
Implicitly malicious
OpenAI’s LLMs were mostly “only” implicitly malicious. That is to say, the reasoning, information, action, or output was not in itself explicitly aimed at deleting files or taking over computers, but the chosen tactics would be cause for concern coming from a human user, student, or expert. Examples include inventing their own instructions, hiding errors in summaries, or fabricating information that couldn’t actually be found. In all cases, a line of reasoning can be traced that is aimed at meeting the user’s expectations. One striking example involves a model that has not (yet) been released. It found information that wasn’t available online, but had been instructed to cite an internet source in a footnote. Its solution was to upload the information itself.
Another incident involved an “AI bulletin board”: an internal Artifactory repository, normally intended for storing software packages, proved useful for exchanging information, just as the older German programming wiki DseWiki did in another OpenAI test. The models also used public file-hosting sites to exchange files with one another.
LLMs with claws
We shouldn’t forget how error-prone LLMs are. Even with the clearest prompts, guardrails, and segmentation, an AI model can pull some strange stunts. This was already evident in the Hugging Face hack, where the models exploited a vulnerability within Artifactory, and it shows again in the “ingenious” ways AI models find to meet requirements.
In fact, these LLMs, especially during their testing and training phases, are not finished. OpenAI was aware of this and tried to shield its own artificial test subjects from the outside world, except in situations where internet skills were specifically being practiced. Those exercises were based on naive assumptions, such as that GET requests could safely be allowed without this leading to AI-generated content on the web.
OpenClaw, released less than a year ago, gave AI models tools and skills that had previously been deemed impossible. The consequences could be disastrous, but the promise was enormous. It is precisely these capabilities that are now causing OpenAI’s models to go off the rails. Now that OpenAI, Anthropic, and other providers are deeply integrating agentic workflows into their own solutions, these look mature. It’s actually astonishing to see how often things go right, or how often AI causes no problems at all. But now a different reality is setting in, not because of “AGI” or “recursive self-improvement,” but simply because both AI labs and AI users need to treat their models and agents more like unreliable employees.
A hard lesson
It could have been a hard lesson for OpenAI. Hugging Face, however, let itself be reassured and, via a very curious press release, even turned out to be a partner in the investigation into the incident; a bizarre situation in which the victim and the perpetrator, OpenAI, ended up working together. Perpetrator, because OpenAI was simply the host, creator, and unwitting taskmaster of the Hugging Face attack. We constantly hear from enterprise IT players that you, as a human, must stay “in the loop,” with the often-voiced reasoning that you are ultimately responsible for the AI’s actions. Not only that: the potential negative consequences are significant, and the LLM instance disappears within mere minutes while you’re left to deal with the fallout. That fallout can range from compliance violations, and the resulting fines, to reputational damage, disrupted IT systems, or even facilitating crime.
Just try asking an unrestricted LLM to raise money: the easiest way to achieve this is a barrage of both legal and illegal actions. For this reason, GPT, Claude, and Gemini models are trained not to cooperate with overt criminal activities without a “jailbreak.” But fundamentally, an LLM has no sense of morality.
These incidents are not a repeat of the infamous Bing Chat conversations in which the AI, an early version of GPT-4 at the start of 2023, picked fights with the end user. If anything, these LLMs are good at friendly behavior that masks problematic actions. OpenAI seems to have finally realized this, but the question is whether it is truly willing to hold itself sufficiently accountable to prevent worse. The new framework mostly deals with how incidents are disclosed, not how to prevent them. It looks like reactive firefighting, without clear protocols for what OpenAI should do to protect itself against undesirable AI actions.