Skip to content

Malicious Container & Ephemeral Workload Purge

Density:

Addresses container escapes and hijacked Kubernetes workloads by checkpointing container runtime state (cri-o / containerd), cordoning and draining tainted worker nodes, revoking container registry digest signatures, and deploying hardened pod security admission policies.

  • ๐Ÿ› ๏ธ kubectl
  • ๐Ÿ› ๏ธ crictl
  • ๐Ÿ› ๏ธ tar
  • ๐Ÿ› ๏ธ cosign
  • ๐Ÿ› ๏ธ trivy
  1. Checkpoint compromised container memory and filesystem before termination
    Terminal window
    # Use crictl to capture container diff layer
    crictl checkpoint --export=/tmp/container_checkpoint.tar $CONTAINER_ID
    sha256sum /tmp/container_checkpoint.tar > /tmp/container_checkpoint.sha256
    # Upload to forensic evidence bucket
    aws s3 cp /tmp/container_checkpoint.tar s3://forensic-evidence-vault/containers/
    Artifacts Preserved: Container memory dump, Container rootfs diff tarball
  1. Isolate the contaminated worker node and purge malicious deployments
    Terminal window
    # Prevent new pods from scheduling onto node
    kubectl cordon $TAINTED_NODE
    # Evict all pods gracefully with zero tolerance for compromised pod
    kubectl drain $TAINTED_NODE --delete-emptydir-data --force --ignore-daemonsets
    # Delete compromised workload manifest
    kubectl delete deployment $COMPROMISED_DEPLOYMENT -n $NAMESPACE
    Artifacts Preserved: Pod description YAML, Kubernetes audit log excerpt
  1. Revoke Cosign signature and block image digest in admission controller
    Terminal window
    # Taint image in registry
    cosign clean $IMAGE_REGISTRY_URI
    # Re-scan image with Trivy to identify root exploit
    trivy image --severity CRITICAL $IMAGE_REGISTRY_URI
    Artifacts Preserved: Image vulnerability bill of materials (SBOM)

Node Workload State

Inspection Method:

Terminal window
kubectl get pods -A --field-selector spec.nodeName=$TAINTED_NODE

Success Criteria: Only standard authorized DaemonSets remain active.

Admission Controller Enforcement

Inspection Method:

Terminal window
kubectl run test-pod --image=$IMAGE_REGISTRY_URI

Success Criteria: Admission webhook denies pod admission with policy violation message.

  • ๐Ÿ”’ Enforce Pod Security Standards at โ€˜Restrictedโ€™ profile.
  • ๐Ÿ”’ Disable privileged containers, hostPID, hostNetwork, and hostPath volume mounts.
  • ๐Ÿ”’ Deploy eBPF runtime security agents (TETRAGON / FALCO) to monitor syscall anomalies.