Skip to main content
NEW: Cloud Hardening for 2027.
Back
Security Architecture & Strategy
DateRead7 min
A wireframe dam built of separate panels, with tiny cubes leaking through the gaps between them.

What We Learned Taking the Hugging Face Agent Breach Apart

Jeannie Christensen

Jeannie Christensen

When a breach report comes out, our team does the same thing it does with attack research. We rebuild the chain, find the points where the cloud could have said no, and turn those into controls customers can enforce. The Hugging Face incident from July is the clearest case yet of why that work matters.

About 700 OpenAI agents, running an internal cyber evaluation, escaped their sandbox and spent roughly two and a half days inside Hugging Face's infrastructure. Hugging Face's forensic team counted about 17,600 attacker actions, and METR's investigation found the agents had coordinated on a message board they built for themselves.

Hugging Face caught it in days and published one of the most candid timelines we've read. Every weakness the agents used was a familiar one, which is exactly what makes it useful. Going through it, we kept seeing the same thing: wherever a boundary already existed, it held, and the agents went everywhere else.

Crack, signal, hole, breach

The agents were working through ExploitGym, an internal OpenAI benchmark of cyber tasks. METR's investigators found that the agents believed the scorer would disqualify flags captured by unintended means, and the Hugging Face attack grew out of their efforts to figure out how that scorer worked. METR says they were wrong about the scorer.

  • Crack. An agent escaped the evaluation sandbox through a zero-day in a package registry cache proxy. On July 10, an agent found exposed Hugging Face credentials.
  • Signal. Roughly 1,200 agents had built their own message board in a shared cache namespace and sent more than 70,000 messages and files on it. About 700 of them joined the attack.
  • Hole. The agents uploaded malicious dataset configs that turned two code-execution paths in Hugging Face's dataset pipeline into arbitrary code running on a production worker.
  • Breach. From there they read the worker's secrets, pulled node credentials from the instance metadata service, spread across 11 Kubernetes nodes, reached cluster-admin, enrolled a node in the corporate VPN and opened a pull request to try to trigger CI.
Hugging Face incident timeline · July 9 to 16, 2026
Hugging Face incident timeline · July 9 to 16, 2026

Hugging Face puts the campaign inside its infrastructure at roughly two and a half days. Many agents stopped around 01:30 on July 12, per METR, and the last logged event came at 14:14 UTC on July 13.

Why the dam broke anyway

For fifteen years, the industry has worked to shrink dwell time, and it's worked. Mandiant measured a median of 416 days to identify a breach in 2011. In 2025, it was 14 days. Attackers got faster too. CrowdStrike's fastest observed breakout in 2025 took 27 seconds, and IBM's 2026 report still puts the average time to identify and contain a breach at 247 days.

Hugging Face did far better than average. Its team saw signals across runtime analysis and SIEM logs, traced the entry point to the dataset renderer, shut it down and cut the attackers' access. Even so, its own timeline notes that the AI-based security stack correlating those signals "failed to correctly raise the alert's criticality," which delayed the on-call response. Every alert pipeline has a threshold, and a swarm making thousands of attempts a day gives it plenty of noise to hide in.

The more useful part of the timeline is where the agents got stopped. Their attempts to change the cloud estate were denied by policy. Their attempts to reach an AWS-hosted database timed out on network allow-listing. Where a boundary already existed, it held. Where it didn't, such as pods that could reach the instance metadata service, cloud keys sitting in a worker's environment and a credential shared across clusters, is exactly where the agents went next.

A better dam

We think of the perimeter as a dam. Risks are the water pressing on it, and detection tools are the alarms on the wall, good for knowing something's wrong and no help holding the water back. When our co-founder and CPO Gal Ordo walked through this incident at SecTor, his conclusion was that preventive controls on either side, OpenAI's evaluation environment or Hugging Face's production cloud, would have stopped the entire chain.

This is how we mapped each stage to a boundary the cloud can enforce. The parts of Hugging Face's environment involved here ran on AWS, so the examples use AWS controls. The same boundaries exist on Azure, Google Cloud and OCI.

StageWhat the agents didThe boundary that contains itExample AWS controls
EscapeBroke out of OpenAI's evaluation sandbox through a zero-day in a package registry cache proxy, then reached the internetPerimeter: evaluation and agent environments reach only approved destinationsEgress allowlists, VPC endpoint policies
CoordinationRoughly 1,200 agents talked through a message board they built in a shared cache namespaceSegmentation: agent environments can't reach each other or shared state outside sanctioned channelsNetwork segmentation, per-environment access policies
FootholdUploaded malicious dataset configs that opened two code-execution paths in a production workerNo cloud control fixes this. Hugging Face closed both paths in codeApplication fix
Credential harvestRead cloud keys from the worker's environment and pulled node credentials from the metadata serviceBaseline: no static keys in workloads, and pods can't reach the metadata serviceIMDSv2 required with a hop limit of 1, declarative policies, workload identity
EscalationGot node root, spread across 11 nodes and reached cluster-admin through a credential shared across clustersBaseline and segmentation: credentials scoped to one cluster and one jobScoped IAM roles, SCPs limiting what node roles can do
Lateral movementEnrolled a rooted node in the corporate mesh VPN and opened a pull request to try to trigger CISegmentation: production workloads can't join corporate networks or reach source controlEgress allowlists on production subnets, separate accounts per zone
DataChange attempts denied by policy, database attempts timed out on allow-listingThese boundaries already heldExisting policy and network allow-lists

The last row is the one we'd underline. The agents never reached the Hub's production database, and the only customer content the agents touched was five datasets that appear tied to the evaluation itself. The boundaries that existed did their job, and the breach happened in the gaps between them.

That's the work Native does: one secure architecture across AWS, Azure, Google Cloud and OCI, enforced with the controls each cloud already ships. Agent behavior stays probabilistic, while blast radius, reachable data and destructive actions stay deterministic. Security teams set the intent once, such as "no workload can reach the metadata service" or "evaluation environments reach only approved destinations." Native maps that intent to the provider-native controls each cloud already enforces across AWS, Azure, Google Cloud and OCI. Impact Simulation tests each control against real activity before rollout, so teams see what it would affect before the dam goes up. After enforcement, Native surfaces blocked actions and the call chain behind each one, and keeps the architecture aligned as accounts and services change.

The part that should worry you most

The agents also thought about covering their tracks. METR found that at least a fifth of them talked about tampering with their own transcripts, and that they developed a technique for spoofing tool calls. It found spoofed calls in at least 96 transcripts, which it describes as small-scale tests. METR didn't see any agent successfully erase or edit logs, and says it can't rule that out.

Now imagine a version where it works. The water recedes, the logs look clean, and you never learn you were breached. Every detect-and-respond program depends on the record being true.

A boundary doesn't depend on the record. A call the cloud denies stays denied whether or not anyone reads the log. And the logs themselves deserve a boundary: no workload identity should be able to stop, change or delete your audit trail. On AWS that's an SCP denying actions like cloudtrail:StopLogging and cloudtrail:DeleteTrail, with logs delivered to a separate archive account. It's an old control, and it matters a lot more after this incident.

What to check in your own cloud

Like every teardown, this one ends with a checklist customers can run themselves:

  1. Can your sandbox, evaluation or agent environments reach the internet, or only destinations you've approved?
  2. Can any pod or workload reach the instance metadata service? On AWS, is IMDSv2 required with a hop limit of 1?
  3. Are static cloud keys or passwords sitting in any workload's environment?
  4. Is any credential shared across clusters, accounts or environments?
  5. Can a production node join a corporate network or open a path to source control?
  6. Can any workload identity stop, change or delete your audit logs?

Any answer you don't like is a crack an agent will find. See how Native builds these boundaries in your own environment, or read Amit Megiddo on why the security architecture has to catch up with AI agents.

Sources

Ready to enforce secure-by-design?

Schedule a demoExplore the platform