Once again, distillation—the process of training AI models on the outputs of other models—is making headlines. Anthropic claims that Alibaba has carried out the largest “adversarial distillation attack” ever. 28.8 million conversations via 25,000 fraudulent accounts are said to have massively siphoned Claude’s AI outputs to the Chinese tech giant. Why is distillation so effective? And is Anthropic’s alarmism justified?
A brief history: distillation, also known as knowledge compression, has been a known concept for twenty years. In 2006, ML expert Rich Caruana and his team discovered that a small classification model could learn from the dataset labels produced by a larger model, and retain much of the same knowledge. Pioneers of modern AI, such as Geoffrey Hinton, subsequently introduced the idea of a “teacher” and “student” model in 2015, whereby the student model could replicate deeper emergent patterns in the “thinking” of the larger model. The effectiveness of this approach was proven rather emphatically in 2019: DistilBERT, a language model that Hugging Face researchers had built based on Google’s BERT, was 40 percent smaller than its larger counterpart but achieved 97 percent of its performance—and it was 60 percent faster to boot.
Modern distillation
The positive perception of distillation persisted for a long time. In fact, in October 2024, OpenAI encouraged developers to distill the AI lab’s leading LLMs via its own API, offering concrete instructions for doing so. The blog post reads like it came from a bizarre alternate universe in 2026. After all, it was OpenAI itself—together with Anthropic—that hassince highlighted the “danger” of model distillation time and again.
A few months later, in January 2025, the success of distillation was demonstrated once again. DeepSeek-R1, a “reasoning” version of the Chinese DeepSeek-V3, exhibited behavior that closely resembled that of OpenAI’s then-groundbreaking o1 model. This led to immense doubts among investors: what if advanced AI becomes commonplace—and dirt cheap? And what if the frontier labs fail to capture the market, losing out to companies that train on their technology?
DeepSeek-R1 itself also proved extremely useful for further distillation, setting up a veritable matryoshka doll sequence of ever-shrinking AI model equivalents. Tiny LLMss, sometimes with as few as 1.5 billion parameters, could suddenly perform astonishingly well with minimal computational power. Whereas GPT-2—which also had 1.5 billion parameters but was released in 2019—had been nothing more than a fun experiment, models at this scale had suddenly become effective for real-world tasks.
Anti-distillation
The tide had turned and distillation was now an economic threat. To make the massive AI investments profitable, AI labs like Anthropic and OpenAI need to build a certain “moat.” They must be able to offer products that cannot be replicated purely due to the capital required. But Chinese AI players—from DeepSeek to Moonshot AI, MiniMax, and now Alibaba—are working against that effort.
In February of this year, Anthropic stated that three Chinese labs, using 24,000 accounts and 16 million prompts, bombarded Claude with distillation attempts. Now, Anthropic itself is claiming that Alibaba has carried out an “adversarial distillation campaign” on an unprecedented scale.
That’s not to say the campaign was necessarily successful. The most recent LLMs have detection mechanisms against distillation. But instead of a direct block or an account ban, Claude secretly helps defend Anthropic’s turf. When faced with a distillation attempt, Fable 5, for example, conceals how it “really” arrives at an answer and is deliberately unhelpful. This is in addition to the fact that Anthropic, OpenAI, and Google no longer fully disclose their reasoning steps; these tokens contain too much valuable information and are now effectively a black box.
The value of distillation remains
Not every AI model is constructed in the same way. Nevertheless, the concept of distillation is highly appealing to AI model developers to generate scaled versions at various pricing and performance tiers. After building Mythos and Fable, Anthropic can reduce these massive LLMs to an Opus, Sonnet, or Haiku. This way, it offers more capabilities than before while using the same computational power as previous models of equivalent size.
Distillation is also extremely useful for open-source AI. A range of LLMs that can run on 8 GPUs, 2 GPUs, 1 GPU, or within the RAM of a standard PC is essential for the practical, widespread adoption of AI. The most complex tasks remain best suited for the largest models, but these can delegate to smaller ones when necessary. This will almost certainly become the paradigm within which AI adoption matures.
Conclusion: An Unstable Stalemate
U.S. AI players recognize that their margins will continue to be threatened by open-source competitors. Distillation—even if it’s becoming more difficult and, as a result, more messy—is here to stay. Chinese companies are violating the terms of service of AI APIs by distilling and are using accounts that are “fraudulent”, or at the very least, fake, to do so. But that is all it is. Unlike many cyber threats, “adversarial distillation” is only a major problem for AI vendors, at least in the short term.
In the longer term, distillation could prevent AI labs from ever recouping the billions of dollars they are currently investing. This could have very negative consequences for the global economy, since economic growth, particularly in the U.S., is concentrated among the major tech players driving AI expansion. If that expansion never becomes profitable, the much-discussed AI bubble could burst.
However, we shouldn’t necessarily feel too much sympathy for Anthropic, OpenAI, and Google. As has been observed time and again, they have ruthlessly devoured the entire internet for their own gain. They certainly didn’t ask politely to do this. The adversarial stance against content producers hasn’t come without a fight; lawsuits are still pending to determine whether this massive AI training has actually produced legally defensible products.
Nevertheless, the genie is out of the bottle, and AI models are here to stay. What’s striking, however, is that it’s precisely those distilled models that are guaranteed to stick around, even if the unthinkable happens and advanced AI comes to a standstill. If the big three American AI model builders remain stuck at a certain LLM performance level, their Chinese competitors will continue to gain ground. As long as this happens with open-weight models that utilize distillation, it ensures that AI remains accessible as a technology.
Read also: Claude Fable 5 and Mythos 5 blocked: is AI now too dangerous?