Skip to main content
Back
IntrusionAWS · Kubernetes
2 min read

Cryptomining on JW Player’s development and staging clusters

Victim
JW Player
When
Initial access date unreportedDisclosed Sep 20, 2018
Entry
Internet access to an unauthenticated internal monitoring dashboard.

An intruder used JW Player’s development and staging servers to mine cryptocurrency after a monitoring dashboard became reachable from the internet. Kubernetes had reversed a manual firewall change intended to keep the dashboard private. Production stayed online, and JW Player found no customer data exposure.

What happened

Outside the clusters
Internet client
Internet
Development and staging
Cluster hosts

Replaying the JW Player intrusion

  1. 1

    The monitoring dashboard becomes public

    Before discovery

    An AWS load balancer admitted internet traffic to the unauthenticated Weave Scope dashboard. The cluster controller had reversed an engineer’s manual restriction on its security group.

  2. 2

    A privileged monitoring agent exposes host capacity

    Day 3

    JW Player traced the miner running on its development and staging nodes to the monitoring agent, which had been deployed with privileged access to each host.

Impact

The miner consumed one CPU core on every development and staging node. JW Player reported uninterrupted service and no access to customer data.

Could this have been prevented?

Keep the dashboard’s network restrictions in its deployment configuration

For a dashboard used only by staff, declare the approved source networks in Kubernetes so the AWS load balancer keeps the same rules after a deployment. On current clusters, teams that can replace a privileged monitoring agent can also enforce the Baseline Pod Security Standards before the replacement runs.

Incident control assessments

1
Prevented
1
Contained
0
Not covered

How it could have been prevented

The same replay with the controls in place. Green marks where a step is refused, blue where it is bounded, and red where no control reaches it. Open a row to read the control, why it acts, and its caveat.

Outside the clusters
Internet client
Internet
Development and staging
Cluster hosts

Replaying the JW Player intrusion

1 · The monitoring dashboard becomes public

1 Contained

2 · A privileged monitoring agent exposes host capacity

1 Prevented

Key facts

  • The dashboard had no sign-in requirement.
  • AWS logs showed that Kubernetes had reversed a manual change to the load balancer’s security group.
  • The miner used one full CPU core on each affected node.
  • Responders isolated one node for investigation and replaced the cluster nodes.
  • JW Player found no access to the Kubernetes API during the intrusion.

Sources

Every fact in this breakdown comes from the public reporting below. The replay is a model of that reporting, not a reconstruction of the environment it describes.

  1. 1.
    How A Cryptocurrency Miner Made Its Way onto Our Internal Kubernetes ClustersBrian Choy, JW Player DevOps · Sep 20, 2018 · Vendor reportResponding engineer’s firsthand investigation, observed resource use and recovery.
  2. 2.
    Security group rulesAmazon Web Services · undated · Vendor reportInbound source restrictions and the union of rules in attached security groups.
  3. 3.
    Pod Security AdmissionKubernetes · undated · Vendor reportNamespace enforcement, exemptions and the boundary at pod admission.
  4. 4.
    Pod Security StandardsKubernetes · undated · Vendor reportThe Baseline profile prohibits privileged containers and hostPath volumes.
  5. 5.
    Kubernetes v1.25: Pod Security Admission Controller in StableKubernetes · Aug 25, 2022 · Vendor reportRelease history for the present-day admission control.

The estate in the replay is illustrative and drawn at the level of attack stages; the victim's real environment is not public. Verdicts describe what a control does when it is in place before the incident. They are not an assessment of the victim's security program.

More cloud attack breakdowns

All breakdowns