Skip to main content
Back
DemonstrationGoogle Cloud · Kubernetes
7 min read

Unit 42's ModeLeak escalation and model exfiltration in Google Cloud Vertex AI

Environment
Unit 42's own Vertex AI test environment in a Google Cloud project; no company was named and no real-world exploitation is reported
When
Not stated; staged before November 12, 2024Disclosed Nov 12, 2024
Entry
An identity holding aiplatform.customJobs.create in the source project (chain 1), and an identity able to export, re-import and deploy a model in the Vertex AI Model Registry (chain 2)

Unit 42 showed that an identity holding only aiplatform.customJobs.create could run its own container image as the AI Platform Custom Code Service Agent, take that agent's credentials from the metadata server, and read and export the source project's buckets and BigQuery datasets, then download images from internal Google repositories. In a second chain, a model re-imported with a swapped container image opened a shell in the prediction tenant project, where a read-only service account listed GKE clusters, pulled the images of other deployed models and copied the tuned LLMs' adapter files; Kubernetes writes and Docker use were refused. Unit 42 says Google implemented fixes for both issues; no source gives a fix date or says what changed.

What the demonstration did

Outside parties
Unit 42
Google
Internet
Metadata serverCustom job VM
Source project
Vertex AI
Prediction tenant project
+ custom job VM
+ metadata server
+ user-data script
Vertex AI
Training tenant project
+ Artifact Registry
+ internal images
+ per-repository access
Google internal repositories
Unit 42 researcher
Custom Code Service Agent
Buckets and BigQuery
Model Registry
Poisoned model server
Prediction service account
Tenant GKE clusters
Tenant image repositories
caip buckets

Replaying Unit 42's ModeLeak demonstration / published November 2024; the run dates are not stated

  1. 1

    A custom job runs as the service agent

    Not dated · before November 12, 2024

    With only aiplatform.customJobs.create, a custom job runs a researcher-built image as the Custom Code Service Agent, and its metadata server hands out that agent's credentials and the VM's user-data script.

  2. 2

    Source project data and internal images

    Not dated · before November 12, 2024

    The service agent's token lists, reads and exports the source project's buckets and BigQuery datasets, and the user-data script leads to images in internal Google repositories.

  3. 3

    A poisoned model in the prediction tenant

    Not dated · before November 12, 2024

    A model re-imported with a swapped container image starts a shell when deployed, and the read-only prediction service account lists its permissions and the tenant's GKE clusters.

  4. 4

    Model images and adapters copied

    Not dated · before November 12, 2024

    Cluster credentials give read-only kubectl access in two tenant clusters, the images of other deployed models are pulled and exported, and the caip buckets with the tuned LLMs' adapter files are copied.

Impact

In Unit 42's test environment, one custom job permission reached the source project's buckets, BigQuery datasets and service account list and images in internal Google repositories; a poisoned model reached the prediction tenant's clusters, the images of every deployed model and the tuned LLMs' adapter files. Kubernetes writes and Docker use were refused.

Outcome

What ModeLeak shows about your managed ML identities

One permission to create a custom job was enough, because the job ran as a Google-managed service agent with broad rights in your project. On your Google Cloud projects, a custom constraint that requires a service account on custom jobs keeps them off that service agent, and an IAM deny policy on the project's service agents and a VPC Service Controls perimeter bound what its token reads. An IAM deny policy on model upload and deploy narrows who can place a model in serving. No customer setting reaches the metadata server, the clusters, the repositories or the buckets of Google's tenant projects, where most of the chain ran. Google implemented fixes there, and no source says what it changed or when.

Would the controls have stopped it?

In hindsight, no control would have stopped it. 6 controls would have limited how far it went. 6 steps have no cloud-provider control. The counts below are control assessments, not attack stages: several can address the same step. The same replay with the controls in place before the incident shows where each phase would have been refused or bounded.

0
Prevented
6
Contained
6
Not covered

How it could have been prevented

The same replay with the controls in place. Green marks where a step is refused, blue where it is bounded, and red where no control reaches it. Open a row to read the control, why it acts, and its caveat.

Outside parties
Unit 42
Google
Internet
Metadata serverCustom job VM
Source project
Vertex AI
Prediction tenant project
+ custom job VM
+ metadata server
+ user-data script
Vertex AI
Training tenant project
+ Artifact Registry
+ internal images
+ per-repository access
Google internal repositories
Unit 42 researcher
Custom Code Service Agent
Buckets and BigQuery
Model Registry
Poisoned model server
Prediction service account
Tenant GKE clusters
Tenant image repositories
caip buckets

Replaying Unit 42's ModeLeak demonstration / published November 2024; the run dates are not stated

1 · A custom job runs as the service agent

2 Contained1 Not covered3 on other clouds

2 · Source project data and internal images

2 Contained1 Not covered3 on other clouds

3 · A poisoned model in the prediction tenant

2 Contained1 Not covered3 on other clouds

4 · Model images and adapters copied

3 Not covered

Key facts

  • One permission, aiplatform.customJobs.create, was enough to start chain 1: the custom job ran a researcher-built image as the AI Platform Custom Code Service Agent and opened a reverse shell, per Unit 42
  • The job's metadata server returned the service agent's credentials and the VM's user-data script, which exposed VM creation details and metadata about internal Google Artifact Registry repositories
  • With the service agent's token, the researchers listed, read and exported Cloud Storage buckets and BigQuery datasets their own identity could not reach, and downloaded images from internal Google repositories; the authors could not fully measure that last impact
  • A model exported from the Model Registry, with the container image in its environment.json swapped, was re-imported and deployed, and started a shell in the prediction tenant project as the custom-online-prediction service account, which the authors describe as read-only
  • Unit 42's write-up of November 12, 2024 reported 3 GKE clusters in the tenant project and read-only kubectl access to 2 of them; the researchers pulled the images of other deployed models and copied caip buckets holding the tuned LLMs' adapter files, whose usability was not shown
  • Kubernetes create, delete, update, attach and exec attempts failed, and Docker could not run; Unit 42 says Google implemented fixes for both issues, and no fix date, CVE or Google statement is published

Sources

Every fact in this breakdown comes from the public reporting below. The replay is a model of that reporting, not a reconstruction of the environment it describes.

  1. 1.
    ModeLeak: Privilege Escalation to LLM Model Exfiltration in Vertex AIUnit 42 (Palo Alto Networks) · Nov 12, 2024 · ResearchCustom job escalation chain, poisoned model chain, service agent and service account identities, permissions used, data and models reached, what held, vendor fix, recommendations
  2. 2.
    Google Vertex AI Vulnerability Exposes Customer DataeWeek · Nov 27, 2024 · NewsRe-report of both flaws and the patch claim

The estate in the replay follows the researchers' description of their own test environment. Verdicts describe what a control does when it is in place before the run. They are not an assessment of any company's security program.

More cloud attack breakdowns

All breakdowns