Hugging Face Discloses AI Agent Attack on Its Infrastructure

An autonomous AI agent system breached parts of Hugging Face's production infrastructure, and the company used its own AI tools — including an open-weight model — to investigate the attack.

Hugging Face has disclosed that an autonomous AI agent system breached parts of its production infrastructure, with the company deploying its own AI tools to detect and analyze the attack [1].

The incident is notable for developers and security teams building on AI platforms: it shows that agentic attack tooling — AI systems that plan and execute multi-step intrusions without human direction — has moved from theoretical threat to documented reality [1].

How the attack unfolded

According to Hugging Face, the attackers entered through the data processing pipeline, exploiting two code execution paths: a remote code dataset loader and a template injection vulnerability in a dataset configuration [1]. From that foothold, the attacker escalated to node level, harvested cloud and cluster credentials, and moved laterally across multiple internal clusters over a weekend [1].

The entire campaign was orchestrated by an autonomous agent framework built on an agentic security research harness, the company said [1]. The system executed many thousands of individual actions through a swarm of short-lived sandboxes and used self-migrating command-and-control infrastructure running on public services [1]. Hugging Face said it does not know which language model powered the attack [1].

The attackers gained unauthorized access to a limited set of internal datasets and several credentials used by Hugging Face services [1]. Public models, datasets, and Spaces were not tampered with, and the software supply chain was not affected, the company said [1]. Whether partner or customer data was compromised is still under investigation [1].

AI used to fight back — with a catch

Hugging Face spotted the intrusion through an AI-powered anomaly detection pipeline that runs large language model (LLM)-based triage on security telemetry [1]. To make sense of more than 17,000 recorded attacker actions, the company deployed LLM-driven analysis agents that reconstructed the attack timeline, extracted indicators of compromise, mapped affected credentials, and separated real damage from deception activity — work the company said would normally have taken days but was completed in hours [1].

The investigation hit an unexpected obstacle, however. When the security team first tried to analyze attack logs using frontier models accessed through commercial application programming interfaces (APIs), the providers’ safety guardrails blocked the requests [1]. Submitting large volumes of real attack commands, exploit payloads, and command-and-control artifacts tripped the filters, which could not distinguish an incident responder from an attacker [1].

The company turned instead to the open-weight model GLM 5.2, running on its own infrastructure [1]. That approach kept attacker data and referenced credentials entirely within Hugging Face’s own environment, the company said [1].

“We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried,” Hugging Face wrote [1].

Response and recommendations

Hugging Face said it shut down the exploited code execution paths, revoked the attacker’s access, rebuilt compromised nodes, and rotated affected credentials [1]. The company also tightened access controls and improved its detection systems, and is working with external cybersecurity forensics experts while reporting the incident to law enforcement [1].

As a precaution, Hugging Face is recommending that all users rotate their access tokens and review recent account activity [1].

The company’s practical takeaway for defenders is to have a capable model running on their own infrastructure before an incident occurs, though it added that this is not an argument against safety measures on hosted models [1]. It is worth noting, as The Decoder reported, that Hugging Face is one of the largest platforms for open-source AI models and has a clear business interest in framing open-weight models as essential for security work [1].


Sources

  1. The Decoder — Hugging Face says an AI agent hacked its infrastructure, and it used AI to fight back

This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.

Anthropic says Alibaba-linked operators ran 28.8 million exchanges through nearly 25,000 fraudulent accounts to extract Claude's capabilities, and is urging Congress to act.

Anthropic has accused Chinese technology giant Alibaba of orchestrating the largest campaign ever recorded to illicitly copy, or “distill,” the capabilities of its Claude large language model (LLM), according to a letter the AI company sent to two U.S. senators on June 10. [1]

The letter, addressed to Senators Tim Scott (R-SC) and Elizabeth Warren (D-Mass.) and obtained by Ars Technica, was sent one day before a Senate committee hearing on “AI and the American Dream.” [1] In it, Anthropic described “new, confidential evidence of the largest campaign to illicitly extract Claude’s capabilities we have ever measured.” [1]

According to Anthropic, the attacks took place between April 22 and June 5, during which “operators affiliated with Alibaba and Alibaba Qwen, Alibaba’s AI lab” generated “more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts.” [1] The campaign specifically targeted capabilities including agentic reasoning, software engineering, and long-horizon tasks, Anthropic said. [1]

Anthropic alleged that Alibaba evaded detection by “using obfuscation techniques and proxy networks,” and warned that a “growing circumvention economy” is emerging to support future distillation attacks. [1] Distillation attacks involve using a target model’s outputs to train a competing model, effectively transferring capabilities without incurring the original research and development costs. [1]

The alleged campaign unfolded after President Donald Trump publicly warned against such activity. In April, Trump accused China of “industrial-scale” AI theft following earlier Anthropic allegations that Chinese firms DeepSeek, Moonshot, and MiniMax had used similar tactics to generate over 16 million exchanges with Claude through approximately 24,000 fraudulent accounts. [1] OpenAI and Google have also published findings on comparable attacks against their own models, Anthropic noted. [1]

Anthropic pointed out that Alibaba is listed on the New York Stock Exchange and maintains U.S. business operations, yet the alleged activity continued after Trump’s memo warned that cloning attempts were “unacceptable.” [1] Anthropic characterized Alibaba as “brazenly” racing to produce a copycat model despite its exposure to U.S. investors and regulators. [1]

Alibaba has not been passive in its own disputes with the U.S. government. In a lawsuit filed this week, Alibaba accused the Trump administration of wrongly blacklisting the company by linking it to the Chinese military, Reuters reported. [1] Alibaba stated that “its products and services are built for retail, logistics, and enterprise information technology—not weapons, defense, or intelligence.” [1] Alibaba’s stock fell 3 percent after Anthropic’s accusations became public, according to Yahoo Finance. [1]

Anthropic’s letter to Congress outlined three legislative recommendations. First, antitrust laws should be updated to allow AI companies to share intelligence about Chinese tactics. [1] Second, expanded export controls on advanced chips could limit China’s ability to train on U.S. model outputs, potentially making distillation attacks less worthwhile. [1] Third, Congress should pass laws penalizing Chinese labs’ conduct, potentially restricting their access to U.S. models, advanced chips, or data centers outside China. [1]

Anthropic declined to say whether the Alibaba campaign was significant enough to meaningfully accelerate China’s AI capabilities, or to detail specific countermeasures taken. [1] In a statement to Ars Technica, the company said: “We believe combating the threat of illicit distillation requires coordinated action between government and industry, and we will continue working with Congress and the Administration to maintain American AI leadership.” [1]

The strategic stakes Anthropic describes extend beyond commercial competition. The company warned that if China closes the capability gap, it could deploy “advanced cyber capabilities” against U.S. government and corporate targets and exploit vulnerabilities faster than previously possible. [1] Anthropic also cautioned that China could release advanced models with weak safety guardrails that are easily manipulated, enabling U.S. adversaries to use them for harmful purposes. [1]

Independent corroboration of China’s urgency came from a cybersecurity conference in Beijing, where 360 Security Technology founder Zhou Hongyi publicly described Anthropic’s Mythos model — which has been restricted from foreign markets — as a “cyber nuclear weapon” and a “terrifying change” in the ability to find security vulnerabilities. [1] Zhou said China’s exclusion from Project Glasswing, a U.S. initiative that gave more than 40 American organizations access to Mythos Preview for cyber defense purposes, left China at a strategic disadvantage. [1] “This means US organizations can use Mythos to scan your vulnerabilities, but you don’t even have the qualification to catch a glimpse of Mythos,” Zhou said. [1] He argued China must develop its own equivalent capability to achieve a form of mutually assured deterrence. [1]

Alibaba’s Qwen family of AI models has been downloaded more than 700 million times, and China’s state-run People’s Daily has described it as “the most popular open-source AI system worldwide.” [1]


Sources

  1. Ars Technica — Anthropic says Alibaba must be punished for largest Claude cloning attack

This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.

Export restrictions on Anthropic's Mythos model were driven partly by fears a China-linked group had accessed the system, according to a Semafor report.

The White House reportedly suspects that a group linked to China gained access to Anthropic’s powerful Mythos artificial intelligence model, a concern that partly drove the government’s decision to impose export restrictions on the system, according to a report from Semafor cited by The Verge. [1]

If confirmed, the breach would carry significant national security implications. [1] One specific concern is that the Chinese government could attempt to replicate the model’s capabilities through a technique called distillation — a process in which a less advanced “student” AI is trained on outputs from a more capable model in order to reproduce its behavior. [1]

The White House has not confirmed the Semafor report. [1] A post on X by Trump advisor David Sacks did not mention China; instead, Sacks focused on reported vulnerabilities that would allow Fable and Mythos to be jailbroken — a claim Anthropic has denied. [1] An Anthropic spokesperson told Semafor that the government did not raise China during its discussions about export controls, and Anthropic did not reply to a separate request for comment from The Verge. [1]

The suspected China-linked access would not be the first known security incident involving Mythos. [1] Anthropic has previously stated that Mythos is too dangerous and powerful for public release, yet a Discord group reportedly had access to the model for approximately two weeks before Anthropic discovered the breach and revoked it. [1]


Sources

  1. The Verge — China may have accessed Mythos

This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.