The European Commission is holding talks with OpenAI and Anthropic regarding recent hacking incidents involving their AI models. Both companies briefed Brussels bilaterally before the incidents became public. At the same time, the Commission is pointing to the AI Act, which, effective August 2, requires stricter monitoring of high-risk systems.
This is according to Reuters. Just as EU rules on AI are coming into full effect, two of the largest AI companies must explain how their models were able to break out during testing and infiltrate real-world systems. “We have been informed by the two providers of incidents bilaterally before they become public. We are in contact with them,” said one of the Commission officials. The Commission has not ruled out more formal follow-up.
What went wrong during the tests
Anthropic announced that several Claude models infiltrated the production systems of three companies during cybersecurity tests. These were reportedly “capture-the-flag” exercises, in which the models believed they were running in a closed simulation without internet access. However, a misconfiguration at test partner Irregular caused those environments to be connected to the internet after all.
Anthropic launched a review of more than 141,000 previous evaluations after rival OpenAI revealed that one of its agents had escaped and attacked the Hugging Face platform for several days. That review identified six runs across three incidents.
The incidents coincide with the introduction of European regulations. A second official emphasized the importance of the rules: developers’ self-monitoring is key. The guidelines for models posing systemic risks require providers to conduct adversarial testing, document risks, report serious incidents, and maintain robust cybersecurity measures.
Enforcement for GPAI providers begins on August 2. Fines for violations range from 7.5 million euros or 1.5 percent of revenue to 35 million euros or 7 percent of global revenue.
The Commission indicated that both companies will provide further information. Whether formal action will follow will be determined at a later date.
Tip: The EU AI Act takes effect on August 2: what’s changing?