Running kube-bench on Kubernetes

The CIS Benchmark has tests for both master and worker nodes. Therefore, the full scope of the test can only be completed on self-managed clusters where you have control over the master nodes. In this recipe, you will learn how to run kube-bench directly on the master and worker nodes.

Let's perform the following steps to run the CIS recommended tests:

1. Download and install the kube-bench command-line interface on one of your master nodes and one of your worker nodes:

$ curl --silent --location "https://github.com/aquasecurity/kube-bench/releases/download/v0.1.0/kube-bench_0.1.0_linux_amd64.tar.gz" | tar xz -C /tmp
$ sudo mv /tmp/kube-bench /usr/local/bin

2. SSH into your Kubernetes master node and run the following command. It will quickly return the result of the test with an explanation and a list of additional manual tests that are recommended to be run after. Here, you can see that 31 checks passed and 36 tests failed:

$ kube-bench master
...
== Summary ==
31 checks PASS
36 checks FAIL
24 checks WARN
1 checks INFO

3. To save the results, use the following command. After the test is complete, move the kube-bench-master.txt file to your localhost for further review:

$ kube-bench master > kube-bench-master.txt

4. Review the content of the kube-bench-master.txt file. You will see the status of the checks from the CIS Benchmark for the Kubernetes guide, similar to the following:

[INFO] 1 Master Node Security Configuration
[INFO] 1.1 API Server
[PASS] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Not Scored)
[FAIL] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Not Scored)
[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
[FAIL] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
[FAIL] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
...

Tests are split into categories that have been suggested in the CIS Benchmark guidelines, such as API Server, Scheduler, Controller Manager, Configuration Manager, etcd, General Security Primitives, and PodSecurityPolicies.

5. Follow the methods suggested in the Remediations section of the report to fix the failed issues and rerun the test to confirm that the correction has been made. You can see some of the remediations that were suggested by the preceding report here:

== Remediations ==
1.1.2 Follow the documentation and configure alternate mechanisms for authentication. Then,
edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.manifest
on the master node and remove the --basic-auth-file=<filename>
parameter.

1.1.5 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.manife$
on the master node and remove the --insecure-bind-address
parameter.

1.1.6 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.manife$
apiserver.yaml on the master node and set the below parameter.
--insecure-port=0

1.1.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.manife$
on the master node and set the below parameter.
--profiling=false

1.2.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.manifest
file on the master node and set the below parameter.
--profiling=false
...

6. Let's take one of the issues from the preceding list. 1.2.1 suggests that we disable the profiling API endpoint. The reason for this is that highly sensitive system information can be uncovered by profiling data and the amount of data and load that's created by profiling your cluster could be put out of service (denial-of-service attack) by this feature. Edit the kube-scheduler.manifest file and add --profiling=false right after the kube-schedule command, as shown in the following code:

...
spec:
  containers:
  - command:
    - /bin/sh
    - -c
    - mkfifo /tmp/pipe; (tee -a /var/log/kube-scheduler.log < /tmp/pipe & ) ; exec
      /usr/local/bin/kube-scheduler --profiling=False --kubeconfig=/var/lib/kube-scheduler/kubeconfig
      --leader-elect=true --v=2 > /tmp/pipe 2>&1
...

7. Run the test again and confirm that the issue on 1.2.1 has been corrected. Here, you can see that the number of passed tests has increased from 31 to 32. One more check has been cleared:

$ kube-bench master
...
== Summary ==
32 checks PASS
35 checks FAIL
24 checks WARN
1 checks INFO

8. Run the test on the worker nodes by using the following command:

$ kube-bench node
...
== Summary ==
9 checks PASS
12 checks FAIL
2 checks WARN
1 checks INFO

9. To save the results, use the following command. After the test has completed, move the kube-bench-worker.txt file to your localhost for further review:

$ kube-bench node > kube-bench-worker.txt

10. Review the content of the kube-bench-worker.txt file. You will see the status of the checks from the CIS Benchmark for the Kubernetes guide, similar to the following:

[INFO] 2 Worker Node Security Configuration
[INFO] 2.1 Kubelet
[PASS] 2.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
[FAIL] 2.1.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
[PASS] 2.1.3 Ensure that the --client-ca-file argument is set as appropriate (Scored)
...

Similarly, follow all the remediations until you've cleared all the failed tests on the master and worker nodes.