What the demonstration did
Replaying Permiso's Bedrock honeypot demonstration / staged in Permiso's own AWS account, June to August 2024
- 1
A bait key on GitHub
Jun 25, 2024Permiso places a long-lived AWS access key and its secret in a public GitHub file on purpose, and actors use the key from the internet within minutes.
- 2
Finding and opening the models
Jun 25 – Aug 6, 2024Actors from twelve ASNs check which Bedrock models the key can use and call the console's own model access APIs; whether those requests opened the models is not stated.
- 3
The quarantine gap and the hijack
Aug 1–6, 2024AWS attaches its quarantine policy to the key on August 1, but the policy denies no Bedrock action; from August 5 one actor runs more than 75,000 successful model invocations in two days.
- 4
Blocked and reported
Aug 6 and 15, 2024AWS removes the account's ability to launch additional Bedrock resources on August 6, and Permiso reports the findings to Anthropic and AWS on August 15.
Actors used the deliberately leaked key from June 25 to August 6, 2024 and ran more than 75,000 successful model invocations in two days for a sexual roleplay service. The quarantine policy AWS applied on August 1 did not cover Bedrock, and AWS added Bedrock actions to it on October 2.
What the honeypot shows about your AWS keys
A long-lived access key that reaches a public repository is used within minutes, and AWS's quarantine is reactive: in August 2024 it denied no Bedrock action. What you control is what the key can do before anyone finds it. In the member accounts of an organization, a service control policy can refuse new long-term access keys and IAM users' calls that carry no MFA, leave Amazon Bedrock off the services an account may use, deny model access requests outside approved administrators, though a model already enabled stays usable, and allow invocations only for approved models and from your own networks. GitHub push protection can refuse a push that carries the key, unless the pusher can bypass it. When AWS restricts an abused account, and what its quarantine denies, is AWS's call.
Would the controls have stopped it?
In hindsight, no control would have stopped it. 7 controls would have limited how far it went. 3 steps have no cloud-provider control. The counts below are control assessments, not attack stages: several can address the same step. The same replay with the controls in place before the incident shows where each phase would have been refused or bounded.
- 0
- Prevented
- 7
- Contained
- 3
- Not covered
How it could have been prevented
The same replay with the controls in place. Green marks where a step is refused, blue where it is bounded, and red where no control reaches it. Open a row to read the control, why it acts, and its caveat.
Replaying Permiso's Bedrock honeypot demonstration / staged in Permiso's own AWS account, June to August 2024
1 · A bait key on GitHub
3 Contained1 Not covered6 on other clouds2 · Finding and opening the models
2 Contained6 on other clouds3 · The quarantine gap and the hijack
2 Contained1 Not covered3 on other clouds4 · Blocked and reported
1 Not coveredKey facts
- Permiso placed a long-lived AWS access key and its secret in a file on GitHub on June 25, 2024 to simulate what had occurred at earlier clients; within minutes, actors tried the key against Amazon Bedrock and other services
- Between June 25 and August 6, 2024, actors from twelve ASNs called nine Bedrock APIs, among them PutUseCaseForModelAccess, CreateFoundationModelAgreement and PutFoundationModelEntitlement; whether those requests opened the models or Permiso enabled them before the leak is not stated
- AWS found the exposed key on June 25 and again on August 1, 2024, when it attached the AWSCompromisedKeyQuarantineV2 policy; the policy then denied no Bedrock action, and AWS added five Bedrock actions to it on October 2, 2024
- Over two days from August 5, 2024, the honeypot recorded more than 75,000 successful model invocations, almost all from one IP address and mostly against Claude Sonnet
- On August 6, 2024 AWS removed the account's ability to launch additional Bedrock resources, about 35 hours after the invoke attempts reached the thousands and about 42 days after the key was first used maliciously
- Krebs on Security reported a $3,500 AWS bill for what Permiso called a two-day experiment; Permiso's own timeline runs from June 25 to August 6, 2024
Sources
Every fact in this breakdown comes from the public reporting below. The replay is a model of that reporting, not a reconstruction of the environment it describes.
- 1.When AI Gets Hijacked: Exploiting Hosted Models for Dark RoleplayingPermiso (P0 Labs) · Oct 3, 2024 · ResearchHoneypot setup, leak on GitHub, Bedrock API sequence, model access request APIs, invocation count, source of the invocations, AWS notices and quarantine, AWS restriction, AWS statement
- 2.A Single Cloud Compromise Can Feed an Army of AI Sex BotsKrebs on Security · Oct 3, 2024 · NewsLeak on GitHub, key used within minutes, bill for the experiment, quarantine restrictions and Bedrock
- 3.AWSCompromisedKeyQuarantineV2 (AWS Managed Policy reference)Amazon Web Services · Oct 2, 2024 · AdvisoryQuarantine policy content, Bedrock actions in the policy, edit date
The estate in the replay follows the researchers' description of their own test environment. Verdicts describe what a control does when it is in place before the run. They are not an assessment of any company's security program.