You're reading from Kubernetes - A Complete DevOps Cookbook

Product typeBook

Published inMar 2020

PublisherPackt

ISBN-139781838828042

Edition1st Edition

Tools

Kubernetes

Concepts

DevOps

Author (1)

Murat Karslioglu

Observability and Monitoring on Kubernetes

In this chapter, we will discuss the built-in Kubernetes tools and the popular third-party monitoring options for your containerized DevOps environment. You will learn how to monitor metrics for performance analysis, and also how to monitor and manage the real-time cost of Kubernetes resources.

By the end of this chapter, you should have knowledge of the following:

Monitoring in Kubernetes
Inspecting containers
Monitoring using Amazon CloudWatch
Monitoring using Google Stackdriver
Monitoring using Azure Monitor
Monitoring Kubernetes using Prometheus and Grafana
Monitoring and performance analysis using Sysdig
Managing the cost of resources using Kubecost

Technical requirements

The recipes in this chapter assume that you have deployed a functional Kubernetes cluster following one of the recommended methods described in Chapter 1, Building Production-Ready Kubernetes Clusters.

Kubernetes' command-line tool, kubectl, will be used for the rest of the recipes in this chapter since it's the main command-line interface for running commands against Kubernetes clusters. We will also use Helm where Helm charts are available to deploy solutions.

Monitoring in Kubernetes

In this section, we will configure our Kubernetes cluster to get core metrics, such as CPU and memory. You will learn how to monitor Kubernetes metrics using the built-in Kubernetes tools both in the CLI and on the UI.

Getting ready

Make sure you have a Kubernetes cluster ready and kubectl configured to manage the cluster resources.

Clone the k8sdevopscookbook/src repository to your workstation to use the manifest files in the chapter8 directory:

$ git clone https://github.com/k8sdevopscookbook/src.git
$ cd /src/chapter8

The Monitoring metrics using Kubernetes Dashboard recipe requires Kubernetes Dashboard v2.0.0 or later to function. If you want to add metric functionality to the dashboard, make sure that you have Kubernetes Dashboard installed by following the instructions in the Deploying Kubernetes Dashboard recipe in Chapter 1, Building Production-Ready Kubernetes Clusters.

How to do it…

This section is further divided into the following subsections...

Inspecting containers

In this section, we will troubleshoot problems related to pods stuck in Pending, ImagePullBackOff, or CrashLoopBackOff states. You will learn how to inspect and debug pods that are having deployment problems in Kubernetes.

Getting ready

Make sure you have a Kubernetes cluster ready and kubectl configured to manage the cluster resources.

How to do it…

This section is further divided into the following subsections to make the process easier:

Inspecting pods in Pending status
Inspecting pods in ImagePullBackOff status
Inspecting pods in CrashLoopBackOff status

Inspecting pods in Pending status

When you deploy applications on Kubernetes, it is inevitable that soon or later you will need to get more information on your application. In this recipe, we will learn to inspect common pods problem of pods stuck in Pending status:

In the /src/chapter8 folder, inspect the content of the mongo-sc.yaml file and deploy it running the following command. The deployment...

Monitoring using Amazon CloudWatch

In this section, we will use Amazon CloudWatch Container Insights to monitor, isolate, and diagnose your containerized applications and microservices environments. As a DevOps or Systems Engineer, you will learn how to use Amazon ECS CloudWatch metrics to monitor service health status and current alarms using automated dashboards that summarize the performance and health of your Amazon EKS clusters by pod, node, namespace, and services.

Getting ready

Clone the k8sdevopscookbook/src repository to your workstation to use manifest files in the chapter8 directory:

$ git clone https://github.com/k8sdevopscookbook/src.git
$ cd src/chapter8/

Make sure you have an Amazon EKS Kubernetes cluster ready and kubectl configured to manage the cluster resources. If you don't already have one you can follow the instructions in Chapter 1, Building Production-Ready Kubernetes Clusters, in the Configuring a Kubernetes cluster on Amazon Web Services section.

How to...

Monitoring using Google Stackdriver

In this section, we will use Google Stackdriver Kubernetes Engine Monitoring to monitor, isolate, and diagnose your containerized applications and microservices environments. You will learn how to use Stackdriver Kubernetes Engine Monitoring to aggregate logs, events, and metrics from your Kubernetes environment on Google Kubernetes Engine (GKE) to help you understand your application's behavior in production.

Getting ready

Make sure you have a GKE cluster ready and kubectl configured to manage the cluster resources. If you don't already have one, you can follow the instructions in Chapter 1, Building Production-Ready Kubernetes Clusters, in the Configuring a Kubernetes cluster on Google Cloud Platform recipe.

How to do it…

This section is further divided into the following subsections to make the process easier:

Installing Stackdriver Kubernetes Engine Monitoring support for GKE
Configuring a workspace on Stackdriver
Monitoring...

Monitoring using Azure Monitor

In this section, we will use Azure Monitor to monitor, isolate, and diagnose your containerized applications and microservices environments. You will learn how to use Azure Monitor to aggregate logs, events, and metrics from your Kubernetes environment on Azure Kubernetes Service (AKS) to help you understand your application's behavior in production.

Getting ready

Make sure you have an AKS cluster ready and kubectl configured to manage the cluster resources. If you don't already have one, you can follow the instructions in Chapter 1, Building Production-Ready Kubernetes Clusters, in the Configuring a Kubernetes cluster on Google Cloud Platform recipe.

How to do it…

This section is further divided into the following subsections to make the process easier:

Enabling Azure Monitor support for AKS using the CLI
Monitoring AKS performance metrics using Azure Monitor
Viewing live logs using Azure Monitor

Enabling Azure Monitor support for AKS...

Monitoring Kubernetes using Prometheus and Grafana

In this section, we will deploy Prometheus and Grafana on our Kubernetes cluster. You will learn how to monitor a Kubernetes service with Prometheus and use Grafana dashboards to visualize cluster and application metrics.

Getting ready

Clone the k8sdevopscookbook/src repository to your workstation to use manifest files in the chapter8 directory:

$ git clone https://github.com/k8sdevopscookbook/src.git
$ cd /src/chapter8

Make sure you have a Kubernetes cluster ready and kubectl configured to manage the cluster resources.

How to do it…

This section is further divided into the following subsections to make the process easier:

Deploying Prometheus Operator using Helm charts
Monitoring metrics using Grafana dashboards
Adding a Grafana dashboard to monitor applications

Deploying Prometheus using Helm charts

Prometheus is a popular open source solution for event monitoring and alerting. Prometheus records real-time metrics in a time...

Monitoring and performance analysis using Sysdig

In this section, we will use Sysdig Monitor to monitor and simplify Kubernetes troubleshooting. You will learn how to install Sysdig Monitor and extend Prometheus functionality to meet more advanced enterprise needs.

Getting ready

All operations mentioned here require a Sysdig account. If you don't have one, go to https://sysdig.com/sign-up/ and create a trial or full account.

For this recipe, we need to have a Kubernetes cluster ready and the Kubernetes command-line tools kubectl and helm installed to manage the cluster resources.

How to do it…

This section is further divided into the following subsections to make the process easier:

Installing the Sysdig agent
Analyzing application performance

Installing the Sysdig agent

Sysdig Monitor is a tool for monitoring and troubleshooting applications available as part of the Sysdig Cloud Native Visibility and Security Platform. In this recipe, you will learn to deploy Sysdig Monitor...

Managing the cost of resources using Kubecost

In this section, we will install and configure the open source Kubecost project, which gives you cost-related visibility into your Kubernetes resources. You will learn how to monitor resource costs to reduce spending and potentially prevent resource-based outages.

Getting ready

This recipe requires a functional Kubernetes cluster deployed on AWS or GCP. Currently, other cloud providers are not supported.

Before you can execute the commands in the following recipes, you need to have kubectl and helm installed. You can find the instructions to install Helm in Chapter 2, Operating Applications on Kubernetes, in the Deploying workloads using Helm charts section.

How to do it…

This section is further divided into the following subsections to ease the process:

Installing Kubecost
Accessing Kubecost dashboard
Monitoring Kubernetes resource cost allocation

Installing Kubecost

Kubecost creates Kubernetes resource-granular models of current...

The rest of the chapter is locked

You have been reading a chapter from

Kubernetes - A Complete DevOps Cookbook

Published in: Mar 2020Publisher: PacktISBN-13: 9781838828042

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Author (1)

Murat Karslioglu

Murat Karslioglu is a distinguished technologist with years of experience using infrastructure tools and technologies. Murat is currently the VP of products at MayaData, a start-up that builds data agility platform for stateful applications, and a maintainer of open source projects, namely OpenEBS and Litmus. In his free time, Murat is busy writing practical articles about DevOps best practices, CI/CD, Kubernetes, and running stateful applications on popular Kubernetes platforms on his blog, Containerized Me. Murat also runs a cloud-native news curator site, The Containerized Today, where he regularly publishes updates on the Kubernetes ecosystem.
Read more about Murat Karslioglu

Other recommended products

Related to this chapter

kubectl: Command-Line Kubernetes in a Nutshell

Kubernetes is a de facto container orchestration system. To manage Kubernetes, you must understand how to work with kubectl, its command-line tool that lets you interact with your clusters to deploy applications and manage their lifecycle. This book is a comprehensive introduction for those who are new to Kubernetes management via the command line.

BookNov 2020136 pages

Kubernetes in Production Best Practices

Kubernetes is a portable, extensible, and open source platform used for orchestrating and managing workloads and services. This book provides detailed coverage of cloud industry standards and best practices and serves as a reference guide for building and operating production-ready Kubernetes infrastructure.

BookMar 2021292 pages

Kubernetes Cookbook

Kubernetes is one of the most popular, sophisticated, and fast-evolving container orchestrators. In this book, you’ll learn the essentials and find out about the advanced administration in Kubernetes. We’ll take you through a step-by-step hands-on approach, which will familiarize you with the Kubernetes ecosystem.

BookMay 2018554 pages

Mastering Service Mesh

Service Mesh helps overcome the operational challenges of connecting, securing, controlling, and observing modern microservices deployment. This book shows you exactly how to use a Service Mesh architecture to manage and operationalize your microservices-based applications.

BookMar 2020626 pages

Hands-On Kubernetes on Azure

This book will help readers to Deploy web applications securely in Microsoft Azure with docker container and having the need for clustering services to achieve high availability, dynamic scalability, and to monitor applications

BookMar 2019258 pages

The DevOps 2.3 Toolkit

Viktor Farcic’s latest book, The DevOps 2.3 Toolkit: Kubernetes, will take you on a hands-on journey with Viktor into the world of Kubernetes, and the tools not only behind the official project but also the wide-range of third-party apps that are available for you to use.

BookSep 2018418 pages

Cloud Native with Kubernetes

Cloud Native with Kubernetes will guide you effectively - from your first steps using Kubernetes right up to running enterprise-grade cloud native applications with best practices. The book covers every aspect of deploying, securing, and operating modern cloud native applications on Kubernetes.

BookJan 2021446 pages

The DevOps 2.4 Toolkit

The DevOps Toolkit 2.4 is a deep exploration of continuous delivery and deployment in Kubernetes using Jenkins. It shows readers how to build, test, and deploy applications in Kubernetes using fully automated Jenkins pipelines.

BookNov 2019398 pages

Learn Kubernetes Security

Kubernetes is a widely used opensource orchestration platform for managing container applications. Developers are not yet aware of the pitfalls of a containerized environment. This book takes you through core security principles, best practices, and real-world use cases to learn mitigation or prevention strategies from known attacks and CVEs.

BookJul 2020330 pages

DevOps with Kubernetes

This book will guide you from container basic concepts to orchestrating containerized applications in Kubernetes. You’ll learn about the Kubernetes basic architecture, components, resources, admission control, and extensions. We’ll show you how to utilize Kubernetes services in three top cloud providers.

BookJan 2019484 pages

Kubernetes Design Patterns and Extensions

Kubernetes is an open source platform that automates scaling, deployment, and management of container-based applications. The book starts by explaining basics about container orchestration and then moves on to explain how the components of a complex system, such as Kubernetes, interact with each other and helps develop vital skills for troubleshooting Kubernetes clusters and applications running in Kubernetes clusters. Once you are done reading the book, you can easily develop and deploy your own applications with Kubernetes in no time.

BookSep 2018106 pages

Kubernetes for Developers

Kubernetes provides a means to describe what your application needs and how it should run by orchestrating containers on your behalf to operate your software across a single, dozens, or hundreds of machines. Originally created and open sourced from Google, you can use their technology to help run, test, and scale your applications regardless of which cloud provider you use, or even running it locally on your own hardware. This book will show you how to take advantage of this amazing technology by introducing the key concepts and walking you through examples in both NodeJS and Python.

BookApr 2018374 pages

Personalised recommendations for you

Based on your interests and search pattern

Designing and Implementing Microsoft Azure Networking Solutions

Designing and Implementing Microsoft Azure Networking Solutions Exam Ref AZ-700 is an all-encompassing guide to the AZ-700 exam and contains all the information you need to succeed in the world of virtual networking with Azure. With this book, you will be fully prepared for the exam and the world of cloud networking.

BookAug 2023524 pages

Microsoft 365 Security, Compliance, and Identity Administration

The Microsoft 365 Security, Compliance, and Identity Administration is a comprehensive guide that helps you employ Microsoft 365's robust suite of features and empowers you to optimize your administrative tasks.

BookAug 2023630 pages

Zero Trust Overview and Playbook Introduction

Get started on Zero Trust with this step-by-step playbook and learn everything you need to know for a successful Zero Trust journey with tailored guidance for every role, covering strategy, operations, architecture, implementation, and measuring success. This book will become an indispensable reference for everyone in your organization.

BookOct 2023240 pages

The Self-Taught Cloud Computing Engineer

This self-study book helps you master multiple clouds, including AWS, Azure, and GCP, and serves as a roadmap to becoming a certified cloud computing expert. The book will guide you to develop a professional cloud career by helping you build a broad cloud knowledge base, developing hands-on cloud computing skills, and getting cloud certified.

BookSep 2023472 pages

Technology Operating Models for Cloud and Edge

This book will help you build and create ownership of a technology operating model, as well as connect your leadership with engineering and operations, keeping your internal and external customers in mind. It provides practical tips on why, where, and how to make the cloud and edge platform paradigm sing for you, your team, and your organization.

BookAug 2023228 pages

Azure Architecture Explained

Azure is the preferred platform to build mission-critical and secure apps. This book provides comprehensive coverage of essential Azure products, services, and solutions vital for every solution architect's success. Elevate your knowledge and master the critical components of Azure to excel in your role with Azure Architecture Explained.

BookSep 2023446 pages

Pentesting Active Directory and Windows-based Infrastructure

This practical guide helps you explore the pentesting of Microsoft infrastructure in detail, and enhances your offensive skillset by showing you the different ways to perform security assessment. This book will help blue teamers and IT engineers get up to speed with possible security issues they may encounter in their Windows environments.

BookNov 2023360 pages

Practical Ansible

In Practical Ansible, you'll work with the latest release of Ansible and learn to solve complex issues quickly with the help of task-oriented scenarios. You'll start by installing and configuring Ansible to automate monotonous and repetitive IT tasks and get to grips with concepts such as playbooks, inventories, plugins, collections, and network modules.

BookSep 2023420 pages

Windows 11 for Enterprise Administrators

Microsoft’s launch of Windows 11 is a step toward satisfying the enterprise administrator’s needs for better management and enhanced user experience customization. This book provides the enterprise administrator with the knowledge needed to fully utilize the advanced feature set of Windows 11 Enterprise.

BookOct 2023286 pages

The Linux DevOps Handbook

This book is for software and IT professionals seeking knowledge on Linux systems and DevOps practices. This book will provide you with guidance and tools to learn and gain proficiency in managing Linux-based infrastructures and knowledge of DevOps.

BookNov 2023428 pages2