Enabling Process Communication with Kube API Through Service Accounts

Humans are not the only ones who rely on a cluster. Processes in containers often need to invoke Kube API as well. When using RBAC for authentication, we need to decide which users will have permissions to perform specific actions. The same holds true for the processes running inside containers.

When we (humans) try to access a Kubernetes cluster with RBAC enabled, we are authenticated as users. Our username provides an identity that API server uses to decide whether we are allowed to perform intended actions. Similarly processes running inside containers might also need to access the API. In such cases, they are authenticated as a specific ServiceAccount.

ServiceAccounts provide a mechanism to grant permissions to processes running inside containers. In many ways, ServiceAccounts are very similar to RBAC users...

Creating a cluster

We'll start the hands-on walk-through by entering the directory where we cloned the vfarcic/k8s-specs repository.

All the commands from this chapter are available in the 02-sa.sh (https://gist.github.com/vfarcic/5fdca8e7f7bb426003abf4ad55745807) Gist.

 1  cd k8s-specs
 2
 3  git pull

Next, we'll need a cluster which we can use to experiment with ServiceAccounts. The requirements are the same as those we used in the previous chapter. We'll need Kubernetes version 1.9 or higher as well as nginx Ingress Controller, RBAC, and a default StorageClass. If you didn't destroy it, please continue using the cluster you created in the previous chapter. Otherwise, it should be reasonably fast to create a new one. For your convenience, the Gists and the specs we used before are available here as well.

• docker4mac.sh: Docker for Mac with 2 CPUs, 2 GB RAM...

Configuring Jenkins Kubernetes plugin

We'll start by creating the same Jenkins StatefulSet we used in the previous chapter. Once it's up-and-running, we'll try to use Jenkins Kubernetes plugin(https://github.com/jenkinsci/kubernetes-plugin). If we're successful, we'll have a tool which could be used to execute continuous delivery or deployment tasks inside a Kubernetes cluster.

 1  cat sa/jenkins-no-sa.yml

We won't go through the definition since it is the same as the one we used in the previous chapter. There's no mystery that has to be revealed, so we'll move on and create the resources defined in that YAML.

A note to minishift users
OpenShift does not allow setting fsGroup in the security context, it uses Routes instead of Ingress, and Services accessible through Routes need to be the LoadBalancer type. Due to those changes, I had to prepare...

Exploring the default ServiceAccount

Jenkins might not be the best starting point in our exploration of ServiceAccounts. Too many things are happening that are out of our control. There's too much "magic" hidden behind Jenkins code. Instead, we'll start with something simpler. We'll run kubectl as a Pod. If we manage to make that work, we should have no problem applying the newly acquired knowledge to Jenkins and other similar use-cases we might have.

Unfortunately, there is no kubectl official image (at least not in Docker Hub), so I built one. The definition is in the vfarcic/kubectl (https://github.com/vfarcic/kubectl) GitHub repository. Let's take a quick look.

1  curl https://raw.githubusercontent.com/vfarcic/kubectl/master/Dockerfile

The Dockerfile is so uneventful and straightforward that there's probably no need going through it. It&apos...

Creating ServiceAccounts

Let's take a look at service accounts currently available in the default Namespace.

 1  kubectl get sa

The output is as follows.

NAME    SECRETS AGE
default 1       24m

At the moment, there is only one ServiceAccount called default. We already saw the limitations of that account. It is stripped from (almost) all the privileges. If we check the other Namespaces, we'll notice that all of them have only the default ServiceAccount. Whenever we create a new Namespace, Kubernetes creates that account for us.

A note to minishift users
OpenShift is an exception. Unlike most other Kubernetes flavors, it created a few ServiceAccounts in the default Namespace. Feel free to explore them later when you learn more about ServiceAccounts and their relations to Roles.

We already established that we'll need to create new ServiceAccounts if we are ever to allow...

Configuring Jenkins Kubernetes plugin with ServiceAccounts

Now that we got a grip on ServiceAccounts, it should be relatively straightforward to correct the problem we experienced with Jenkins. As a reminder, we could not configure the Kubernetes plugin. We experienced the same forbidden message as when we tried to use kubectl container with the default ServiceAccount. Now that we know that ServiceAccounts provide permissions to processes running inside containers, all we have to do is to define one for Jenkins.

We'll spice it up a bit with a slightly more complicated use-case. We'll try to run Jenkins master in one Namespace and perform builds in another. That way we can have a clear separation between Jenkins and "random" stuff our builds might be doing. Through such separation, we can guarantee that Jenkins will (probably) not be affected if we do something...

Using ServiceAccounts from side-car containers

We still have one more pending issue that we can solve with ServiceAccounts. In the previous chapter we tried to use cvallance/mongo-k8s-sidecar container in hopes it'll dynamically create and manage a MongoDB replica set.

We failed because, at that time, we did not know how to create sufficient permissions that would allow the side-car to do its job. Now we know better.

Let's take a look at an updated version of our go-demo-3 application.

 1  cat sa/go-demo-3.yml

The relevant parts of the output are as follows

...
apiVersion: v1
kind: ServiceAccount
metadata:
  name: db
  namespace: go-demo-3

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: db
  namespace: go-demo-3
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list"]

---

apiVersion: rbac.authorization...

What now?

ServiceAccounts combined with Roles and RoleBindings are an essential component for continuous deployment or any other process that needs to communicate with Kubernetes. The alternative is to run an unsecured cluster which is not an option for any but smallest organizations. RBAC is required when more than one person is operating or using a cluster. If RBAC is enabled, ServiceAccounts are a must. We'll use them a lot in the chapters that follow.

Please consult the APIs that follow for any additional information about ServiceAccounts and related resources.

• ServiceAccount v1 core (https://v1-9.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.9/#serviceaccount-v1-core)
• Role v1 rbac (https://v1-9.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.9/#role-v1-rbac)
• ClusterRole v1 rbac (https://v1-9.docs.kubernetes.io/docs/reference/generated/kubernetes...