Skip to main content
Back
DemonstrationAzure · Kubernetes
4 min read

WireServing: Mandiant reads AKS cluster secrets from a single pod

Environment
Azure Kubernetes Service clusters with Azure CNI and the Azure network policy, as staged by Mandiant; no company was named
When
Run date not stated – Aug 19, 2024Disclosed Aug 19, 2024
Entry
Command execution in a pod of an affected cluster, which Mandiant says can come from a vulnerable workload, a build job or a compromised developer account; no host network and no root were needed.

Mandiant's WireServing research started from command execution in an ordinary pod of an Azure Kubernetes Service cluster, with no host network and no root. The pod reached two undocumented Azure platform services, WireServer and the host plugin, and decrypted the node's provisioning settings, which held the node's client credentials and a TLS bootstrap token. With the token, the cluster signed a certificate for each node, and as each node the researchers read the secrets of the workloads scheduled on it. Mandiant reported the issue to Microsoft through MSRC, and Microsoft fixed it before the publication; no source gives the dates or says what the fix changed.

What the demonstration did

Research team
Mandiant
Internet
Azure platform servicesWireServer · host plugin
AKS
Workload Pod
AKS
Kubernetes Control Plane
+ node client credentials
+ cluster CA certificate
+ TLS bootstrap token
Cluster Node
Researchers' Code
Pod Network
Kubernetes API
Certificate Signing
Node Authorizer
Workload Secrets

Replaying Mandiant's WireServing demonstration / published August 2024; the run date is not stated

  1. 1

    Pod reaches the node's platform services

    Not dated · published Aug 19, 2024

    From an ordinary pod, WireServer hands out the settings key and the host plugin returns the node's settings, which decrypt to the node credentials and a TLS bootstrap token.

  2. 2

    Node credentials, then every workload secret

    Not dated · published Aug 19, 2024

    The node credentials list the nodes, the bootstrap token gets a certificate signed for each node, and each node identity reads its workloads' secrets.

  3. 3

    Disclosure and fix

    Before Aug 19, 2024 · dates not stated

    Mandiant reports the issue to Microsoft through MSRC, and Microsoft fixes the underlying issue before the publication.

Impact

From a single pod with no special privileges, the chain reached the key that decrypts the node's provisioning settings, took the node credentials and the TLS bootstrap token, had a certificate signed for every node and read the secrets of all running workloads; Microsoft fixed the underlying issue.

Outcome

What the demonstration shows about your AKS clusters

The chain needed only code in one pod. No Azure setting that AKS supports on its nodes refuses a pod's request to WireServer, and the cluster signed bootstrap requests whatever node identity they named. The one control in the chain sits in Kubernetes: a NetworkPolicy that keeps your pods' egress to the destinations they need, on a network plugin that enforces it, which Mandiant names as the control that stops the whole chain. Microsoft fixed the underlying issue; the public sources do not say what the fix changed.

Would the controls have stopped it?

In hindsight, no control would have stopped it. 1 control would have limited how far it went. 6 steps have no cloud-provider control. The counts below are control assessments, not attack stages: several can address the same step. The same replay with the controls in place before the incident shows where each phase would have been refused or bounded.

0
Prevented
1
Contained
6
Not covered

How it could have been prevented

The same replay with the controls in place. Green marks where a step is refused, blue where it is bounded, and red where no control reaches it. Open a row to read the control, why it acts, and its caveat.

Research team
Mandiant
Internet
Azure platform servicesWireServer · host plugin
AKS
Workload Pod
AKS
Kubernetes Control Plane
+ node client credentials
+ cluster CA certificate
+ TLS bootstrap token
Cluster Node
Researchers' Code
Pod Network
Kubernetes API
Certificate Signing
Node Authorizer
Workload Secrets

Replaying Mandiant's WireServing demonstration / published August 2024; the run date is not stated

1 · Pod reaches the node's platform services

1 Contained2 Not covered3 on other clouds

2 · Node credentials, then every workload secret

3 Not covered

3 · Disclosure and fix

1 Not covered

Key facts

  • Mandiant started from command execution in a pod of an AKS cluster that used Azure CNI and the Azure network policy; the pod needed no host networking and no root
  • Building on CyberCX's May 2023 research, the pod requested from WireServer the key that encrypts protected extension settings, and the host plugin returned the node's extension settings
  • The decrypted Custom Script Extension settings held the node's provisioning script with its client key and certificate, the cluster certificate authority certificate and a TLS bootstrap token
  • With the bootstrap token, Mandiant submitted a certificate signing request in each node's identity, and AKS signed it automatically
  • As each node, the Node Authorizer gave the secrets of the workloads scheduled on it; repeated for every active node, this gave the secrets of all running workloads in the cluster
  • Mandiant reported the issue to Microsoft through MSRC, and Microsoft fixed it before the August 19, 2024 publication; no source gives the report date, the fix date or what the fix changed

Sources

Every fact in this breakdown comes from the public reporting below. The replay is a model of that reporting, not a reconstruction of the environment it describes.

  1. 1.
    "WireServing" Up Credentials: Escalating Privileges in Azure Kubernetes ServicesMandiant (Google Cloud Threat Intelligence) · Aug 19, 2024 · Researchthe affected configuration, the assumed initial access, the WireServer key, the HostGAPlugin extension settings, the provisioning script secrets, the node credentials listing nodes, the bootstrap token creating certificate requests, the automatic signing, the Node Authorizer and workload secrets, the disclosure through MSRC, Microsoft's fix, the NetworkPolicy advice
  2. 2.
    Azure SSRF MetadataCyberCX · May 15, 2023 · ResearchWireServer and HostGAPlugin roles, the protected settings key from WireServer, extension settings from the HostGAPlugin, the assumed access (local access or a server-side request forgery)
  3. 3.
    Mandiant uncovers critical privilege escalation vulnerability in Azure Kubernetes serviceSiliconANGLE · Aug 20, 2024 · NewsMicrosoft's fix before publication as reported, the affected configuration, outside comment

The estate in the replay follows the researchers' description of their own test environment. Verdicts describe what a control does when it is in place before the run. They are not an assessment of any company's security program.

More cloud attack breakdowns

All breakdowns