NSA spies ample opportunities to harden Kubernetes

You can trust them, they probably know all the weak spots. Ahem.

If Kubernetes is so complicated that even Google is automating its setup, then it's worth paying attention when U.S. spy giant, the National Security Agency (NSA) points to strengthening it.

Kubernetes, as the NSA points out, provides "several flexibility and security benefits compared to traditional, monolithic software platforms." Unfortunately, that flexibility comes with a lot of moving pieces with their own cybersecurity considerations. 

The NSA and Cybersecurity and Infrastructure Security Agency (CISA) recently updated their Kubernetes Hardening Guidance [PDF] which, while designed for government agencies, is still a great set of recommendations for independent organizations.

The document points to three common sources of compromise in Kubernetes: Supply chain risks coming from acquired infrastructure or container builds, malicious threat actors attempting to breach security, and insider threats both active and passive. 

The NSA and CISA make a lot of recommendations (separated into five categories) and provide extensive documentation on enacting them, along with policy samples. "Kubernetes clusters can be complex to secure and are often abused in compromises that exploit their misconfigurations," the document says.

Consider this a preemptive warning to close these holes before someone else finds them.

Hardening Kubernetes Pods

Pods are the smallest deployable Kubernetes units, and consist of one or more containers with shared resources and specifications.

"Pods are often a cyber actor's initial execution environment upon exploiting a container," the NSA says, so make hardening these a priority, and advising users:

• Use containers that can run apps without root permissions
• When possible, use immutable file systems
• Scan container images for vulnerabilities and misconfigurations
• Enforce a minimum level of security that includes preventing privileged containers, denying use of features often used in breakout attacks, rejecting any containers that execute as root and using security services that harden apps against exploitation.

Hardening a network that hosts Kubernetes

"By default, Kubernetes resources are not isolated and do not prevent lateral movement or escalation if a cluster is compromised," the NSA says in the doc. With that in mind, be sure to:

• Use role-based access control and a firewall to lock down access to control plane nodes
• Put control plane components and nodes on a separate network
• Limit access to Kubernetes etcd server
• Require control plane components to use authenticated TLS certificates
• Encrypt etcd at rest, and use a different TLS cert for communicating
• Establish network policies that isolate resources
• Have an explicit deny network policy in place
• Don't store credentials and sensitive info in config files. Instead, put it in Kubernetes Secrets, and then be sure to encrypt it. 

Ensuring safe logins

There are only a few recommendations here: Disable anonymous logins, require strong authentication, and create role-based policies that are unique for different roles. 

Properly logging and detecting threats

Logging is essential for catching suspicious activity, as in determining the best way to analyze those logs. The NSA suggests the following, starting with enabling logging in the first place: 

• Make sure logs are persistent to ensure availability in the case of a hardware failure
• Configure logging throughout the entire Kubernetes environment, including APIs, cluster metrics, Pod seccomp and others
• Aggregate logs to a location external to the Kubernetes cluster
• Find a monitoring and alerting system suitable for your use case, and use it.

Properly patching

There's no need to guess what comes here: Apply patches promptly, regularly scan for vulnerabilities and perform penetration tests, and be sure to uninstall or delete unused components so they don't become a black box waiting for takeover. ®