Chapter 18. Robust Infrastructure with Kubernetes

In the previous chapter, we used Docker to pre-build and package different parts of our application, such as Elasticsearch and our API server, into Docker images. These images are portable and can be deployed independently onto any environment. Although this revised approach automated some aspects of our workflow, we are still manually deploying our containers on a single server.

This lack of automation presents the risk of human error. Deploying on a single server introduces a single point of failure (SPOF), which reduces the reliability of our application.

Instead, we should provide redundancy by spawning multiple instances of each service, and deploying them across different physical servers and data centers. In other words, we should deploy our application on a cluster.

Clusters allow us to have high availability, reliability, and scalability. When an instance of a service becomes unavailable, a failover mechanism can redirect unfulfilled...

High availability

Availability is a measure of the proportion of time that a system is able to fulfill its intended function. For an API, it means the percentage of time that the API can respond correctly to a client's requests.

Measuring availability

Availability is usually measured as the percentage of time the system is functional (Uptime) over the total elapsed time:

This is typically represented as "nines". For example, a system with an availability level of "four nines" will have an uptime of 99.99% or higher.

Following the industry standard

Generally speaking, the more complex a system, the more things can go wrong; this translates to a lower availability. In other words, it is much easier to have a 100% uptime for a static website than for an API.

So, what is the industry standard for availability for common APIs? Most online platforms offer a service level agreement (SLA) that includes a clause for the minimum availability of the platform. Here are some examples (accurate at the time...

High reliability

Reliability is a measure of the confidence in a system, and is inversely proportional to the probability of failure.

Reliability is measured using several metrics:

• Mean time between failures (MTBF): Uptime/number of failures
• Mean time to repair (MTTR): The average time it takes the team to fix a failure and return the system online

Testing for reliability

The easiest way to increase reliability is to increase test coverage of the system. This is, of course, assuming that those tests are meaningful tests.

Tests increase reliability by:

• Increasing MTBF: The more thorough your tests, the more likely you'll catch bugs before the system is deployed.
• Reducing MTTR: This is because historical test results inform you of the last version which passes all tests. If the application is experiencing a high level of failures, then the team can quickly roll back to the last-known-good version.

High throughput

Throughput is a measure of the number of requests that can be fulfilled in a given time interval.

The throughput of a system depends on several factors:

• Network Latency: The amount of time it takes for the message to get from the client to our application, as well as between different components of the application
• Performance: The computation speed of the program itself
• Parallelism: Whether requests can be processed in parallel

We can increase throughput using the following strategies:

• Deploying our application geographically close to the client: Generally, this reduces the number of hops that a request must make through proxy servers, and thus reduces network latency. We should also deploy components that depend on each other close together, preferably within the same data center. This also reduces network latency.
• Ensure servers have sufficient resources: This makes sure that the CPU on your servers are sufficiently fast, and that the servers have enough memory to perform their...

High scalability

Scalability is a measure of how well a system can grow in order to handle higher demands, while still maintaining the same levels of performance.

The demand may arise as part of a sustained growth in user uptake, or it may be due to a sudden peak of traffic (for example, a food delivery application is likely to receive more requests during lunch hours).

A highly scalable system should constantly monitor its constituent components and identify components which are working above a "safe" resource limit, and scale that component either horizontally or vertically.

We can increase scalability in two ways:

• Scale Vertically or scaling Up: Increase the amount of resources (for example, CPU, RAM, storage, bandwidth) to the existing servers
• Scale Horizontally or scaling out: Adding servers to the existing cluster

Scaling vertically is simple, but there'll always be a limit as to how much CPU, RAM, bandwidth, ports, and even processes the machine can handle. For example, many kernels have...

Clusters and microservices

In order to make our system be highly available, reliable, scalable, and produce high throughput, we must design a system that is:

• Resilient/Durable: Able to sustain component failures
• Elastic: Each service and resource can grow and shrink quickly based on demand

Such systems can be achieved by breaking monolithic applications into many smaller stateless components (following the microservices architecture) and deploying them in a cluster.

Microservices

Instead of having a monolithic code base that caters to many concerns, you can instead break the application down into many services which, when working together, make up the whole application. Each service should:

• Have one or very few concerns
• Be de-coupled from other services
• Be stateless (if possible)

With a monolithic application, all the components must be deployed together as a single unit. if you want to scale your application, you must scale by deploying more instances of the monolith. Furthermore, because there...

Cluster management

Deploying our application in a microservices manner inside a cluster is simple enough in principle, but actually quite complex to implement.

First, you must provision servers to act as nodes inside your cluster. Then, we'll need to set up a handful of tools that work in concert with each other to manage your cluster. These tools can be categorized into two groups:

• Cluster-level tools: Works at the cluster level, and makes global decisions that affect the whole cluster
• Node-level tools: Resides within each node. It takes instructions from, and feedback to, cluster-level tools in order to coordinate the management of services running inside the node.

For the cluster-level tools, you'll need the following:

• A scheduler: This dictates which node a particular service will be deployed on.
• A Discovery Service: This keeps a record of how many instances of each service are deployed, their states (for example, starting, running, terminating and so on.), where they're deployed, and so on...

Picking a cluster management tool

Having to manage these different cluster management components individually is tedious and error-prone. Luckily, cluster management tools exist that provides a common API that allows us to configure these tools in a consistent and automated manner. You'd use the Cluster management tool's API instead of manipulating each component individually.

Cluster management tools are also known as cluster orchestration tools or container orchestration tools. Although there may be slight nuances between the different terms, we can regard them as the same for the purpose of this chapter.

There are a few popular cluster management tools available today:

• Marathon (https://mesosphere.github.io/marathon/): By Mesosphere and runs on Apache Mesos.
• Swarm (https://docs.docker.com/engine/swarm/): The Docker engine includes a swarm mode that manages Docker containers in clusters called swarms. You may also group certain containers together using Docker Compose.
• Kubernetes: The de facto...

Control Planes and components

The components we described previously—scheduler, Discovery Service, Global Configuration Store, and so on—are common to all Cluster Management Tools that exist today. The difference between them is how they package these components and abstract away the details. In Kubernetes, these components are aptly named Kubernetes Components.

We will distinguish between generic "components" with Kubernetes Components by using the capital case for the latter.

In Kubernetes terminology, a "component" is a process that implements some part of the Kubernetes cluster system; examples include the kube-apiserver and kube-scheduler. The sum of all components forms what you think of as the "Kubernetes system", which is formally known as the Control Plane.

Similar to how we categorized the cluster tools into cluster-level tools and node-level tools, Kubernetes categorizes Kubernetes Components into Master Components and Node Components, respectively. Node Components operates within...

Kubernetes objects

Now that you understand the different Components that make up the Kubernetes system, let's shift our attention to Kubernetes API Objects, or Objects (with a capital O), for short.

As you already know, with Kubernetes, you don't need to interact directly with individual Kubernetes Components; instead, you interact with kube-apiserver and the API server will coordinate actions on your behalf.

The API abstracts away raw processes and entities into abstract concepts called Objects. For instance, instead of asking the API server to "Run these groups of related containers on a node", you'd instead ask "Add this Pod to the cluster". Here, the group of containers is abstracted to a Pod Object. When we work with Kubernetes, all we're doing is sending requests to the Kubernetes API to manipulate these Objects.

The four basic objects

There are four basic Kubernetes Objects:

• Pod: A group of closely-related containers that should be managed as a single unit
• Service: An abstraction that proxies...

Setting up the local development environment

Now that you understand the different Components of Kubernetes and the abstractions (Objects) that the API provides, we are ready to migrate the deployment of our application to using Kubernetes. In this section, we will learn the basics of Kubernetes by running it on our local machine. Later on in this chapter, we will build on what we've learned and deploy our application on multiple VPSs, managed by a cloud provider.

Checking hardware requirements

To run Kubernetes locally, your machine needs to fulfill the following hardware requirements:

• Have 2 GB or more of available RAM
• Have two or more CPU cores
• Swap space is disabled

Make sure you are using a machine which satisfies those requirements.

 

Cleaning our environment

Because Kubernetes manages our application containers for us, we no longer need to manage our own Docker containers. Therefore, let's provide a clean working environment by removing any Docker containers and images related to our application...

Creating our cluster

With the Kubernetes daemon (installed and ran by minikube) and the Kubernetes client (kubectl) installed, we can now run minikube start to create and start our cluster. We'd need to pass in --vm-driver=none as we are not using a VM.

Note

If you are using a VM, remember to use the correct --vm-driver flag.

We need to run the minikube start command as root because the kubeadm and kubelet binaries need to be downloaded and moved to /usr/local/bin, which requires root privileges.

However, this usually means that all the files created and written during the installation and initiation process will be owned by root. This makes it hard for a normal user to modify configuration files.

Fortunately, Kubernetes provides several environment variables that we can set to change this.

Setting environment variables for the local cluster

Inside .profile (or its equivalents, such as .bash_profile or .bashrc), add the following lines at the end:

export MINIKUBE_WANTUPDATENOTIFICATION=false
export...

Creating our first Pod

Now that we have a cluster running locally, let's deploy our Elasticsearch service on it. With Kubernetes, all services run inside containers. Conveniently for us, we are already familiar with Docker, and Kubernetes supports the Docker container format.

However, Kubernetes doesn't actually deploy containers individually, but rather, it deploys Pods. As already mentioned, Pods are a type of basic Kubernetes Objects—abstractions provided by the Kubernetes API. Specifically, Pods are a logical grouping of containers that should be deployed and managed together. In Kubernetes, Pods are also the lowest-level unit that Kubernetes manages.

Containers inside the same Pod share the following:

• Lifecycle: All containers inside a Pod are managed as a single unit. When a pod starts, all the containers inside the pod will start (this is known as a shared fate). When a Pod needs to be relocated to a different node, all containers inside the pod will relocate (also known as co-scheduling...

Understanding high-level Kubernetes objects

The more observant of you might have noticed the following output after you ran kubectl:

deployment.apps "elasticsearch" created

When we run kubectl run, Kubernetes does not create a Pod directly; instead, Kubernetes automatically creates a Deployment Object that will manage the Pod for us. Therefore, the following two commands are functionally equivalent:

$ kubectl run <name> --image=<image>
$ kubectl create deployment <name> --image=<image>

To demonstrate this, you can see a list of active Deployments using kubectl get deployments:

$ kubectl get deployments
NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
elasticsearch   1         1         1            1           2s

The benefit of using a Deployment object is that it will manage the Pods under its control. This means that if the Pod fails, the Deployment will automatically restart the Pod for us.

Generally, we should not imperatively instruct Kubernetes to create...

Declarative over imperative

Pods, Deployments, and ReplicaSet are examples of Kubernetes Objects. Kubernetes provides you with multiple approaches to run and manage them.

• kubectl run—imperative: You provide instructions through the command line to the Kubernetes API to carry out
• kubectl create—imperative: You provide instructions, in the form of a configuration file, to the Kubernetes API to carry out

• kubectl apply—declarative: You tell the Kubernetes API the desired state of your cluster using configuration file(s), and Kubernetes will figure out the operations required to reach that state

kubectl create is a slight improvement to kubectl run because the configuration file(s) can now be version controlled; however, it is still not ideal due to its imperative nature.

If we use the imperative approach, we'd be manipulating the Kubernetes object(s) directly, and thus be responsible for monitoring all Kubernetes objects. This essentially defeats the point of having a Cluster Management Tool.

The...

Configuring Elasticsearch cluster

From the output of kubectl describe pods (or kubectl get pod), we can see that the IP address of the Pod named elasticsearch-699c7dd54f-n5tmq is listed as 172.17.0.5. Since our machine is the node that this Pod runs on, we can access the Pod using this private IP address.

 

The Elasticsearch API should be listening to port 9200. Therefore, if we make a GET request to http://172.17.0.5:9200/, we should expect Elasticsearch to reply with a JSON object:

$ curl http://172.17.0.5:9200/
{
  "name" : "CKaMZGV",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "dCAcFnvOQFuU8pTgw4utwQ",
  "version" : {
    "number" : "6.3.2",
    "lucene_version" : "7.3.1"
    ...
  },
  "tagline" : "You Know, for Search"
}

We can do the same for Pods elasticsearch-699c7dd54f-pft9k and elasticsearch-699c7dd54f-pm2wz, which have the IPs 172.17.0.4 and 172.17.0.6, respectively:

$ kubectl get pods -l app=elasticsearch -o=custom-columns=NAME:.metadata.name,IP:.status.podIP
NAME IP...

Deploying on cloud provider

So far, we've deployed everything locally so that you can experiment freely without costs. But for us to make our service available to the wider internet, we need to deploy our cluster remotely, with a cloud provider.

DigitalOcean supports running Kubernetes clusters, and so we will sign in to our DigitalOcean dashboard and create a new cluster.

Creating a new remote cluster

After signing into your DigitalOcean account, click on the Kubernetes tab on your dashboard. You should be greeted with the message Get started with Kubernetes on DigitalOcean. Click on the Create a Cluster button and you will be shown a screen similar to how you configured your droplet:

Make sure you select at least three Nodes, where each node has at least 4 GB of RAM. Then, click Create Cluster. You'll be brought back to the main Kubernetes tab, where you can see that the cluster is being provisioned:

Click on the cluster and you'll be brought to the Overview section for the cluster:

Click on...

Persisting data

However, we're not finished yet! Right now, if all of our Elasticsearch containers fail, the data stored inside them would be lost.

This is because containers are ephemeral, meaning that any file changes inside the container, be it addition or deletion, only persist for as long as the container persists; once the container is gone, the changes are gone.

 

 

This is fine for stateless applications, but our Elasticsearch service's primary purpose is to hold state. Therefore, similar to how we persist data using Volumes in Docker, we need to do the same with Kubernetes.

Introducing Kubernetes Volumes

Like Docker, Kubernetes has an API Object that's also called Volume, but there are several differences between the two.

With both Docker and Kubernetes, the storage solution that backs a Volume can be a directory on the host machine, or it can be a part of a cloud solution like AWS.

And for both Docker and Kubernetes, a Volume is an abstraction for a piece of storage that can be attached...

Introducing PersistentVolume (PV)

To tackle these issues, Kubernetes provides the PersistentVolume (PV) object. PersistentVolume is a variation of the Volume Object, but the storage capability is associated with the entire cluster, and not with any particular Pod.

Consuming PVs with PersistentVolumeClaim (PVC)

When an administrator wants a Pod to use storage provided by a PV, the administrator would create a new PersistentVolumeClaim (PVC) object and assign that PVC Object to the Pod. A PVC object is simply a request for a suitable PV to be bound to the PVC (and thus the Pod).

After the PVC has been registered with the Master Control Plane, the Master Control Plane would search for a PV that satisfies the criteria laid out in the PVC, and bind the two together. For instance, if the PVC requests a PV with at least 5 GB of storage space, the Master Control Plane will only bind that PVC with PVs which have at least 5 GB of space.

After the PVC has been bound to the PV, the Pod would be able to...

Dynamic volume provisioning with StorageClass

To resolve these issues, Kubernetes provides another API Object called StorageClass. With StorageClass, Kubernetes is able to interact with the cloud provider directly. This allows Kubernetes to provision new storage volumes, and create PersistentVolumes automatically.

Basically, a PersistentVolume is a representation of a piece of storage, whereas StorageClass is a specification of how to create PersistentVolumes dynamically. StorageClass abstracts the manual processes into a set of fields you can specify inside a manifest file.

Defining a StorageClass

For example, if you want to create a StorageClass that will create Amazon EBS Volume of type General Purpose SSD (gp2), you'd define a StorageClass manifest like so:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Retain

 

 

Here's what each field means (required fields are marked with an asterik (*...

Visualizing Kubernetes Objects using the Web UI Dashboard

You've been introduced to a lot of Kubernetes in this chapter—Namespaces, Nodes, Pods, Deployments, ReplicaSet, StatefulSet, DaemonSet, Services, Volumes, PersistentVolumes, and StorageClasses. So, let's take a mini-breather before we continue.

So far, we've been using kubectl for everything. While kubectl is great, sometimes, visual tools can help. The Kubernetes project provides a convenient Web UI Dashboard that allows you to visualize all Kubernetes Objects easily.

Note

The Kubernetes Web UI Dashboard is different from the DigitalOcean Dashboard.

Both kubectl and the Web UI Dashboard make calls to the kube-apiserver, but the former is a command-line tool, whereas the latter provides a web interface.

By default, the Web UI Dashboard is not deployed automatically. We'd normally need to run the following to get an instance of the Dashboard running on our cluster:

$ kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard...

Deploying the backend API

We've deployed Elasticsearch, so let's carry on with the rest of the deployment—of our backend API and our frontend application.

The elasticsearch Docker image used in the deployment was available publicly. However, our backend API Docker image is not available anywhere, and thus our remote Kubernetes cluster won't be able to pull and deploy it.

Therefore, we need to build our Docker images and make it available on a Docker registry. If we don't mind our image being downloaded by others, we can publish it on a public registry like Docker Hub. If we want to control access to our image, we need to deploy it on a private registry.

For simplicity's sake, we will simply publish our images publicly on Docker Hub.

Publishing our image to Docker Hub

First, go to https://hub.docker.com/ and create an account with Docker Hub. Make sure to verify your email.

Then, click on Create | create Repository at the top navigation. Give the repository a unique name and press Create. You can...

Creating a backend Service

Next, we should deploy a Service that sits in front of the backend Pods. As a recap, every backend Pod inside the backend Deployment will have its own IP address, but these addresses can change as Pods are destroyed and created. Having a Service that sits in front of these Pods allow other parts of the application to access these backend Pods in a consistent manner.

Create a new manifest file at ./manifests/backend/service.yaml with the following content:

apiVersion: v1
kind: Service
metadata:
  name: backend
  labels:
    app: backend
spec:
  selector:
    app: backend
  ports:

  - port: 8080
    name: api
  - port: 8100
    name: docs

And deploy it using kubectl apply:

$ kubectl apply -f ./manifests/backend/service.yaml
service "backend" created

$ kubectl get services
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)             AGE
backend         ClusterIP   10.32.187.38   <none>        8080/TCP,8100/TCP   4s
elasticsearch   ClusterIP   None...

Exposing services through Ingress

An Ingress is a Kubernetes Object that sits at the edge of the cluster and manages external access to Services inside the cluster.

The Ingress holds a set of rules that takes inbound requests as parameters and routes them to the relevant Service. It can be used for routing, load balancing, terminate SSL, and more.

Deploying the NGINX Ingress Controller

An Ingress Object requires a Controller to enact it. Unlike other Kubernetes controllers, which are part of the kube-controller-manager binary, the Ingress controller is not. Apart from the GCE/Google Kubernetes Engine, the Ingress controller needs to be deployed separately as a Pod.

The most popular Ingress controller is the NGINX controller (https://github.com/kubernetes/ingress-nginx), which is officially supported by Kubernetes and NGINX. Deploy it by running kubectl apply:

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
$ kubectl apply -f https://raw...

Summary

In this chapter, we have successfully deployed our Elasticsearch instance and backend API on Kubernetes. We have learned the roles of each Component and the types of Objects each manages.

You've come a long way since we started! To finish it off, let's see if you can use what you've learned to deploy the frontend application on Kubernetes on your own.