Kubernetes Extensions

Kubernetes is highly customizable and extensible so that any segment of the system can be configured comprehensively and extended with new features. Extension points of Kubernetes do not focus on low-level configuration of the built-in resources, such as pods or stateful sets. However, extending Kubernetes means extending the operations of Kubernetes itself. These extension points enable many practices, including creating new Kubernetes resources, automating Kubernetes and human interactions, and intervening with the creation or editing of resources and their scheduling mechanisms.

In this chapter, extension points and patterns will be presented, and the most common and essential extension points will be covered. Firstly, the Kubernetes API will be enhanced, and human knowledge will be converted into the automation of Kubernetes operators. Secondly, the control...

Kubernetes Extension Points

Kubernetes itself and its built-in resources are highly configurable so that any modern cloud-native application can be configured to run on the cloud environment. When it comes to adding new capabilities, converting human knowledge into code and automating more, the Kubernetes extension comes to the rescue. Fortunately, to extend the capabilities of Kubernetes, users do not need to download the source code, make changes, build and deploy the complete system. With its modularity, the extension points of Kubernetes are already defined and ready to use.

Kubernetes extension points focus on the current functionalities of Kubernetes and its environment. Built-in components and how to extend Kubernetes are summarized in the following categories:

• Kubernetes clients: It is possible to extend client applications such as kubectl by writing kubectl plugins...

Extending Kubernetes Clients

Kubernetes client applications and libraries are the main entry points for accessing the Kubernetes API. With these applications and libraries, it is possible to automate and extend Kubernetes operations.

For the official Kubernetes client applications, kubectl can be extended by writing plugin applications. Some of the most popular plugins enhance the capabilities of kubectl:

• It switches the Kubernetes cluster context automatically
• It calculates and displays the uptime information of pods
• It connects via SSH into a container with a specific user

Official Kubernetes code generators can generate official Kubernetes client libraries and Kubernetes server codes. These generators create the required source code for internal versioned types, clients informers, and protobuf codecs.

With the extension points on client applications and libraries, it is...

Extending the Kubernetes API

Kubernetes already has a rich set of resources, starting from pods as building blocks to higher-level resources such as stateful sets and deployments. Modern cloud-native applications can be deployed in terms of Kubernetes resources and their high-level configuration options. However, they are not sufficient when human expertise and operations are required. Kubernetes enables extending its own API with new resources and operates them as Kubernetes-native objects with the following features:

• RESTful API: New resources are directly included in the RESTful API so that they are accessible with their special endpoints.
• Authentication and authorization: All requests for new resources go through the steps of authentication and authorization, like native requests.
• OpenAPI discovery: New resources can be discovered and integrated into OpenAPI specifications...

Kubernetes Dynamic Admission Control

The Kubernetes API server is responsible for every request. The extension point in the request life cycle in the API server is for dynamic admission control. The admission controller is one of the most important stages of the request life cycle, since it intercepts and checks whether a request should be approved or not.

For every API request, first of all, the requester is checked by authentication and authorization. Afterward, admission controllers are run and decide to approve or reject the request. Finally, validation steps are carried out, and the resulting objects are stored:

Life cycle of a Kubernetes API request

The dynamic part of admission control comes from the fact that they can be dynamically added, removed, or updated during the runtime of Kubernetes clusters. In addition to the built-in admission controllers, there are ways...

Extending the Kubernetes Scheduler

Pods are the basic unit of work that are scheduled by Kubernetes to run on nodes. By default, Kubernetes has a built-in scheduler, and it tries to assign pods to the nodes evenly by ensuring that there are sufficient free resources. There are some use cases to configure and extend the scheduler behavior of Kubernetes considering the custom requirements of scalable and reliable cloud-native applications:

• Running certain pods on specialized hardware
• Co-locating some pods that include interacting services
• Dedicating some nodes to some users

Scheduler customization and extension patterns, starting from the basics to the complex, are listed as follows:

• Assigning node labels and using node selectors
• Using affinity and anti-affinity rules
• Marking nodes with taints, and pods with tolerations
• Creating and deploying custom scheduler algorithms
  

...

Extending Kubernetes Infrastructure

Kubernetes clusters are run on actual bare-metal clusters and interact with the infrastructure systems running on the servers. Extension points for infrastructure are still in the design stage and not mature enough for standardization. However, they can be grouped as follows:

• Server: The Kubernetes node components interact with container runtimes such as Docker. Currently, Kubernetes is designed to work with any container runtime that implements the Container Runtime Interface (CRI) specification. CRI consists of libraries, protocol buffers, and the gRPC API to define the interaction between Kubernetes and the container environment.
• Network: Kubernetes and the container architecture requires high-performance networking, decoupled from container runtime. The connections between containers and network interfaces are defined with the abstraction...

Summary

In this chapter, extending Kubernetes was covered, where we enabled converting domain expertise into automation and intervening Kubernetes operations. Firstly, the extension points in Kubernetes were presented to show its built-in extension
capabilities. Throughout the chapter, new resources were added to the Kubernetes API, and their operations were automated so that Kubernetes can work for custom resources in addition to the built-in ones. Following this, resource creation logic was extended with dynamic admission controllers, and you were shown how to include operational requirements in the Kubernetes API resource life cycle.

Finally, configuring the scheduler of Kubernetes was presented to cover all extensive
requirements for nodes and inter-pod relations. How to write, deploy, and use a custom scheduler was also shown. With the extension capabilities included in...