Reader small image

You're reading from  The Kubernetes Workshop

Product typeBook
Published inSep 2020
PublisherPackt
ISBN-139781838820756
Edition1st Edition
Right arrow
Authors (6):
Zachary Arnold
Zachary Arnold
author image
Zachary Arnold

Zachary Arnold works as a software engineer at Ygrene Energy Fund. Zach has an experience of over 10 years in modern web development. He is an active contributor to the Open Source Kubernetes project in both SIG-Release and SIG-Docs currently focusing on security. He has been running clusters in production since Kubernetes 1.7 and has spoken at the previous 4 KubeCons. His passion areas in the project center on building highly stable Kubernetes cluster components and running workloads securely inside of Kubernetes.
Read more about Zachary Arnold

Sahil Dua
Sahil Dua
author image
Sahil Dua

Sahil Dua is a software engineer. He started using Kubernetes to run machine learning workloads. Currently, he is running various types of applications on Kubernetes. He shared his learnings as a keynote session at KubeCon Europe 2018. He is a passionate open source contributor and has contributed to some famous projects such as Git, pandas, hound, go-GitHub, and so on. He has been an open source community leader for over 2 years at DuckDuckGo.
Read more about Sahil Dua

Wei Huang
Wei Huang
author image
Wei Huang

Wei Huang: Wei works as a senior software engineer in IBM. He has over 10 years' experiences around database, data warehouse tooling, cloud, container, monitoring and devops. He started to use Kubernetes since 1.3, including extending Kubernetes LoadBalancer using CRD, networking, scheduling and monitoring. Now he is a core maintainer of Kubernetes SIG-Scheduling.
Read more about Wei Huang

Faisal Masood
Faisal Masood
author image
Faisal Masood

Faisal Masood is a cloud transformation architect at AWS. Faisal's focus is to assist customers in refining and executing strategic business goals. Faisal main interests are evolutionary architectures, software development, ML lifecycle, CD and IaC. Faisal has over two decades of experience in software architecture and development.
Read more about Faisal Masood

Mélony Qin
Mélony Qin
author image
Mélony Qin

Mélony Y. QIN, also known as CloudMelon, is the founder of CloudMelon Vis, a tech media and educational platform for technopreneurs in the cloud-native and serverless space, and a former product manager at Microsoft. With a passion for cloud-native technologies, OSS, DevOps, Kubernetes, serverless, data, and AI, Mélony has authored multiple books, including the Certified Kubernetes Administrator (CKA) Exam Guide, the Kubernetes Workshop, and Microsoft Azure Infrastructure, all published by Packt Publishing. Mélony is a member of the Association for Computing Machinery (ACM) and the Project Management Institute (PMI), leveraging her extensive experience with diverse cloud technologies to drive innovation in the cloud-native, serverless, and generative AI space. She runs the CloudMelonVis YouTube channel and Cloud-Native Innovators newsletter, read by professionals from top tech companies such as Microsoft, Google, Amazon, Dell, and Carrefour.
Read more about Mélony Qin

Mohammed Abu Taleb
Mohammed Abu Taleb
author image
Mohammed Abu Taleb

Mohammed Abu-Taleb works as a Technical Advisor at Microsoft. Working at Microsoft CSS team for troubleshooting complex issues and cases for premier customers that are using Azure Kubernetes Services (AKS). Prior that, Mohammed was a SME (subject matter expert) for the azure managed monitoring service (Azure Monitor) focusing on designing, deploying, and troubleshooting monitoring strategies for containers.
Read more about Mohammed Abu Taleb

View More author details
Right arrow

The Need for Container Orchestration

The k8s-for-beginners container we built in Exercise 1.01, Creating a Docker Image and Uploading it to Docker Hub, is nothing but a simple demonstration. In the case of a serious workload deployed in a production environment, and to enable hundreds of thousands of containers running in a cluster, we have many more things to consider. We need a system to manage the following problems:

Container Interactions

As an example, suppose that we are going to build a web app with a frontend container displaying information and accepting user requests, and a backend container serving as a datastore that interacts with the frontend container. The first challenge is to figure out how to specify the address of the backend container to the frontend container. It is not a good idea to hardcode the IP, as the container IP is not static. In a distributed system, it is not uncommon for containers or machines to fail due to unexpected issues. So, the link between any two containers must be discoverable and effective across all the machines. On the other hand, the second challenge is that we may want to limit which containers (for example, the backend container) can be visited by which kind of containers (for example, its corresponding frontend ones).

Network and Storage

All the examples that we gave in the previous sections used containers running on the same machine. This is pretty straightforward, as the underlying Linux namespaces and cgroup technologies were designed to work within the same OS entity. If we want to run thousands of containers in a production environment, which is pretty common, we have to resolve the network connectivity issue to ensure that different containers across different machines are able to connect with each other. On the other hand, local or temporary on-disk storage doesn't always work for all workloads. Applications may need the data to be stored remotely and be available to be mounted at will to any machine in the cluster the container is run on, no matter if the container is starting up for the first time or restarting after a failure.

Resource Management and Scheduling

We have seen that a container leverages Linux cgroups to manage its resource usage. To be a modern resource manager, it needs to build an easy-to-use resource model to abstract resources such as CPU, RAM, disk, and GPU. We need to manage a number of containers efficiently, and to provision and free up resources in time so as to achieve high cluster utilization.

Scheduling involves assigning an appropriate machine in the cluster for each of our workloads to run on. We will take a closer look at scheduling as we proceed further in this book. To ensure that each container has the best machine to run, the scheduler (a Kubernetes component that takes care of scheduling) needs to have a global view of the distribution of all containers across the different machines in the cluster. Additionally, in large data centers, the containers would need to be distributed based on the physical locations of the machines or the availability zones of the cloud services. For example, if all containers supporting a service are allocated to the same physical machine, and that machine happens to fail, the service will experience a period of outage regardless of how many replicas of the containers you had deployed.

Failover and Recovery

Application or machine errors are quite common in a distributed system. Therefore, we must consider container and machine failures. When containers encounter fatal errors and exit, they should be able to be restarted on the same or another suitable machine that is available. We should be able to detect machine faults or network partitions so as to reschedule the containers from problematic machines to healthy ones. Moreover, the reconciliation process should be autonomous, to make sure the application is always running in its desired state.

Scalability

As demand increases, you may want to scale up an application. Take a web frontend application as an example. We may need to run several replicas of it and use a load balancer to distribute the incoming traffic evenly among the many replicas of containers supporting the service. To walk one step further, depending on the volume of incoming requests, you may want the application to be scaled dynamically, either horizontally (by having more or fewer replicas), or vertically (by allocating more or fewer resources). This takes the difficulty of system design to another level.

Service Exposure

Suppose we've tackled all the challenges mentioned previously; that's to say, all things are working great within the cluster. Well, here comes another challenge: how can the applications be accessed externally? On one hand, the external endpoint needs to be associated with the underlying on-premises or cloud environment so that it can leverage the infrastructure's API to make itself always accessible. On the other hand, to keep the internal network traffic always going through, the external endpoint needs to be associated with internal backing replicas dynamically – any unhealthy replicas need to be taken out and backfilled automatically to ensure that the application remains online. Moreover, L4 (TCP/UDP) and L7 (HTTP, HTTPS) traffic has different characteristics in terms of packets, and, therefore, needs to be treated in slightly different ways to ensure efficiency. For example, the HTTP header information can be used to reuse the same public IP to serve multiple backend applications.

Delivery Pipeline

From a system administrator's point of view, a healthy cluster must be monitorable, operable, and autonomous in responding to failures. This requires the applications deployed on to the cluster to follow a standardized and configurable delivery pipeline so that it can be managed well at different phases, as well as in different environments.

An individual container is typically used only for completing a single functionality, which is not enough. We need to provide several building blocks to connect the containers all together to accomplish a complicated task.

Orchestrator: Putting All the Things Together

We don't mean to overwhelm you, but the aforementioned problems are very serious, and they arise as a result of the large number of containers that need to be automatically managed. Compared to the VM era, containers do open another door for application management in a large, distributed cluster. However, this also takes container and cluster management challenges to another level. In order to connect the containers to each other to accomplish the desired functionality in a scalable, high-performant, and self-recovering manner, we need a well-designed container orchestrator. Otherwise, we would not be able to migrate our applications from VMs to containers. It's the third reason why containerization technologies began to be adopted on a large scale in recent years, particularly upon the emergence of Kubernetes – which is the de facto container orchestrator nowadays.

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
The Kubernetes Workshop
Published in: Sep 2020Publisher: PacktISBN-13: 9781838820756

Authors (6)

author image
Zachary Arnold

Zachary Arnold works as a software engineer at Ygrene Energy Fund. Zach has an experience of over 10 years in modern web development. He is an active contributor to the Open Source Kubernetes project in both SIG-Release and SIG-Docs currently focusing on security. He has been running clusters in production since Kubernetes 1.7 and has spoken at the previous 4 KubeCons. His passion areas in the project center on building highly stable Kubernetes cluster components and running workloads securely inside of Kubernetes.
Read more about Zachary Arnold

author image
Sahil Dua

Sahil Dua is a software engineer. He started using Kubernetes to run machine learning workloads. Currently, he is running various types of applications on Kubernetes. He shared his learnings as a keynote session at KubeCon Europe 2018. He is a passionate open source contributor and has contributed to some famous projects such as Git, pandas, hound, go-GitHub, and so on. He has been an open source community leader for over 2 years at DuckDuckGo.
Read more about Sahil Dua

author image
Wei Huang

Wei Huang: Wei works as a senior software engineer in IBM. He has over 10 years' experiences around database, data warehouse tooling, cloud, container, monitoring and devops. He started to use Kubernetes since 1.3, including extending Kubernetes LoadBalancer using CRD, networking, scheduling and monitoring. Now he is a core maintainer of Kubernetes SIG-Scheduling.
Read more about Wei Huang

author image
Faisal Masood

Faisal Masood is a cloud transformation architect at AWS. Faisal's focus is to assist customers in refining and executing strategic business goals. Faisal main interests are evolutionary architectures, software development, ML lifecycle, CD and IaC. Faisal has over two decades of experience in software architecture and development.
Read more about Faisal Masood

author image
Mélony Qin

Mélony Y. QIN, also known as CloudMelon, is the founder of CloudMelon Vis, a tech media and educational platform for technopreneurs in the cloud-native and serverless space, and a former product manager at Microsoft. With a passion for cloud-native technologies, OSS, DevOps, Kubernetes, serverless, data, and AI, Mélony has authored multiple books, including the Certified Kubernetes Administrator (CKA) Exam Guide, the Kubernetes Workshop, and Microsoft Azure Infrastructure, all published by Packt Publishing. Mélony is a member of the Association for Computing Machinery (ACM) and the Project Management Institute (PMI), leveraging her extensive experience with diverse cloud technologies to drive innovation in the cloud-native, serverless, and generative AI space. She runs the CloudMelonVis YouTube channel and Cloud-Native Innovators newsletter, read by professionals from top tech companies such as Microsoft, Google, Amazon, Dell, and Carrefour.
Read more about Mélony Qin

author image
Mohammed Abu Taleb

Mohammed Abu-Taleb works as a Technical Advisor at Microsoft. Working at Microsoft CSS team for troubleshooting complex issues and cases for premier customers that are using Azure Kubernetes Services (AKS). Prior that, Mohammed was a SME (subject matter expert) for the azure managed monitoring service (Azure Monitor) focusing on designing, deploying, and troubleshooting monitoring strategies for containers.
Read more about Mohammed Abu Taleb