17. Advanced Scheduling in Kubernetes

Overview

This chapter focuses on scheduling, which is the process by which Kubernetes selects a node for running a Pod. In this chapter, we will take a closer look at this process and the Kubernetes Scheduler, which is the default Kubernetes component responsible for this process.

By the end of this chapter, you will be able to use different ways to control the behavior of the Kubernetes Scheduler to suit the requirements of an application. The chapter will equip you to be able to choose appropriate Pod scheduling methods to control which nodes you want to run your Pods on based on your business needs. You will learn about the different ways to control the scheduling of Pods on the Kubernetes cluster.

Introduction

We have seen that we package our applications as containers and deploy them as a Pod in Kubernetes, which is the minimal unit of Deployment. With the help of the advanced scheduling capabilities provided by Kubernetes, we can optimize the deployment of these Pods with respect to our hardware infrastructure to meet our needs and get the most out of the available resources.

Kubernetes clusters generally have more than a few nodes (or machines or hosts) where the Pod can be executed. Consider that you are managing a few of the machines and you have been assigned to execute an application on these machines. What would you do to decide which machine is the best fit for the given application? Until now in this workshop, whenever you wanted to run a Pod on a Kubernetes cluster, have you mentioned which node(s) the Pod should run on?

That's right – we don't need to; Kubernetes comes with a smart component that finds the best node to run your Pod. This component...

The Kubernetes Scheduler

As mentioned in the introduction, a typical cluster has several nodes. When you create a Pod, Kubernetes has to choose a node and assign the Pod to it. This process is known as Pod scheduling.

The Kubernetes component that is responsible for deciding which node a Pod should be assigned to for execution is called a scheduler. Kubernetes comes with a default scheduler that suffices for most use cases. For example, the default Kubernetes Scheduler spreads the load evenly in the cluster.

Now, consider a scenario in which two different Pods are expected to communicate with each other very often. As a system architect, you may want them to be on the same node to reduce latency and free up some internal networking bandwidth. The Scheduler does not know the relationship between different types of Pods, but Kubernetes provides ways to inform the Scheduler about this relationship and influence the scheduling behavior so that these two different Pods can be hosted...

The Pod Scheduling Process

The scheduler works in a three-step process: filtering, scoring, and assigning. Let's take a look at what happens during the execution of each of these steps. An overview of the process is described in the following diagram:

Figure 17.1: An overview of how the Kubernetes Scheduler selects a suitable node

Filtering

Filtering is a process in which the Kubernetes Scheduler runs a series of checks or filters to see which nodes are not suitable to run the target Pod. An example of a filter is to see if the node has enough CPU and memory to host the Pod, or if the storage volume requested by the Pod can be mounted on the host. If the cluster has no node that's suitable to meet the requirements of the Pod, then the Pod is deemed un-schedulable and is not executed on the cluster.

Scoring

Once the Kubernetes Scheduler has a list of feasible nodes, the second step is to score the nodes and find the best node(s) to host the...

Managing the Kubernetes Scheduler

Kubernetes provides many parameters and objects through which we can manage the behavior of the Kubernetes Scheduler. We will look into the following ways of managing the scheduling process:

• Node affinity and anti-affinity
• Pod affinity and anti-affinity
• Pod priority and preemption
• Taints and tolerations

Node Affinity and Anti-Affinity

Using node affinity rules, a Kubernetes cluster administrator can control the placement of Pods on specific sets of nodes. Node affinity or anti-affinity allows you to constrain which nodes a Pod can run on based on the labels of the nodes.

Imagine that you are an administrator of the shared Kubernetes cluster in a bank. Multiple teams are running their applications on the same cluster. Your organization's security group has identified nodes that can run data-sensitive applications and would like you to make sure that no other applications run on those nodes. Node affinity or anti...

Pod Affinity and Anti-Affinity

Pod affinity and Pod anti-affinity allow your Pods to check what other Pods are running on a given node before they are scheduled on that node. Note that other Pods in this context do not mean a new copy of the same Pod, but Pods related to different workloads.

Pod affinity allows you to control on which node your Pod is eligible to be scheduled based on the labels of the other Pods that are already running on that node. The idea is to cater to the need to place two different types of containers relative to each other at the same place or to keep them apart.

Consider that your application has two components: a frontend part (for example, a GUI) and a backend (for example, an API). Let's assume that you want to run them on the same host because the communications between frontend and backend Pods would be faster if they are hosted on the same node. By default, on a multi-node cluster (not Minikube), the Scheduler will schedule such Pods on different...

Pod Priority

Kubernetes allows you to associate a priority with a Pod. If there are resource constraints, if a new Pod with high priority is requested to be scheduled, the Kubernetes scheduler may evict the Pods with lower priority in order to make room for the new high-priority Pod.

Consider an example where you are a cluster administrator and you run both critical and non-critical workloads in the cluster. An example is a Kubernetes cluster for a bank. In this case, you would have a payment service as well as the bank's website. You may decide that processing payments are of higher importance than running the website. By configuring Pod priority, you can prevent lower-priority workloads from impacting critical workloads in your cluster, especially in cases where the cluster starts to reach its resource capacity. This technique of evicting lower-priority Pods to schedule more critical Pods could be faster than adding additional nodes and would help you better manage traffic...

Taints and Tolerations

Previously, we have seen how Pods can be configured to control which node they run on. Now we will see how nodes can control which Pods can run on them using taints and tolerations.

A taint prevents the scheduling of a pod unless that Pod has a matching toleration for the Pod. Think of taint as an attribute of a node and a toleration is an attribute of a Pod. The Pod will get scheduled on the node only if the Pod's toleration matches the node's taint. The taints on a node tell the scheduler to check which Pods tolerate the taint and run only those Pods that match their toleration with the node's taint.

A taint definition contains the key, value, and effect. The key and value will match the Pod toleration definition in the Pod specification, while the effect instructs the scheduler what should be done once the node's taint matches the Pod's toleration.

The following diagram provides an overview of how the process of controlling...

Using a Custom Kubernetes Scheduler

Building your own fully featured scheduler is out of the scope of this workshop. However, it is important to understand that the Kubernetes platform allows you to write your own scheduler if your use case requires it, although it is not recommended to use a custom scheduler unless you have a very specialized use case.

A custom scheduler runs as a normal Pod. You can specify in the definition of the Pod running your application to use the custom scheduler. You can add a schedulerName field in the Pod specification with the name of the custom scheduler as shown in this sample definition:

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-custom-scheduler
spec:
  containers:
    - name: mutating-pod-example-container
      image: k8s.gcr.io/busybox
      command: [ "/bin/sh", "-c", "while :; do echo '.'; sleep...

Summary

The Kubernetes Scheduler is a powerful software that abstracts the work of selecting the appropriate node for a Pod on a cluster. The Scheduler watches for unscheduled Pods and attempts to find suitable nodes for them. Once it finds a suitable node for a Pod, it updates etcd (via the API server) that the Pod has been bound to the node.

The scheduler has matured with every release of Kubernetes. The default behavior of the scheduler is sufficient for a variety of workloads, although you have also seen many ways to customize the way that the Scheduler associates resources with Pods. You have seen how node affinity can help you schedule Pods on your desired nodes. Pod affinity can help you schedule a Pod relative to another Pod, and it is a good tool for applications where multiple modules are targeted to be placed next to each other. Taints and tolerations can also help you assign specific workloads to specific nodes. You have also seen that Pod priority can help you schedule...