Hugging Face has been the target of a breach carried out entirely by an army of AI agents. The platform detected and analyzed the attack largely using an AI model. This is the first time Hugging Face has experienced something like this.
Hugging Face detected unauthorized access to a limited number of internal datasets and to various credentials for its services. Public models, datasets, and Spaces remained unaffected, the company reported in its July 16 announcement. Container images and published packages were also found to be clean.
The attack was orchestrated from start to finish by an autonomous agent framework. Hugging Face reached this conclusion based on the nature of the attack. The attacker carried out many thousands of individual actions via a “swarm” of short-lived sandboxes, using a self-migrating command-and-control system hosted on public services.
From a malicious dataset to entire clusters
The breach began in the data-processing pipeline, where Hugging Face acknowledges that AI platforms are particularly vulnerable. A malicious dataset exploited two code-execution paths: a remote-code dataset loader and a template injection in a dataset configuration. This allowed the attacker to run code on a processing worker, then escalate to the node level, collect cloud and cluster credentials, and move laterally across multiple internal clusters over the course of a weekend.
Hugging Face has not previously encountered this specific type of attack, but is no stranger to (attempted) infiltrations. Two years ago, it discovered a suspicious breach on its Spaces platform, and a year before that, hackers were able to manipulate popular LLMs via exposed API tokens. The platform was also exploited earlier this year to distribute Android malware. This time, the platform itself is the victim.
Guardrails hindered the investigation
During the analysis, Hugging Face ran into an unexpected problem. Frontier models behind commercial APIs blocked the submission of real exploit payloads and C2 artifacts because their safety guardrails do not distinguish between an incident responder and an attacker. The company therefore ran the forensic analysis on GLM 5.2, an open-weight model, on its own infrastructure. This ensured that the data did not leave its own environment.
These guardrails are absent in Mythos 5 and GPT-5.6-Cyber, models that Anthropic and OpenAI offer to partners in their security programs. At Anthropic, this is called Project Glasswing; at OpenAI, it’s called Daybreak. This creates a somewhat curious divide between the “in-crowd” that can use the very strongest LLMs for cybersecurity purposes and a much larger group that relies on open-weight models without restrictions, though, at the time of writing, these models offer slightly weaker performance.