Search icon CANCEL
Subscription
0
Cart icon
Cart
Close icon
You have no products in your basket yet
Save more on your purchases!
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Monitoring Docker
Monitoring Docker

Monitoring Docker: Monitor your Docker containers and their apps using various native and third-party tools with the help of this exclusive guide!

By Russ McKendrick
$35.99 $24.99
Book Dec 2015 148 pages 1st Edition
eBook
$35.99 $24.99
Print
$43.99
Subscription
$15.99 Monthly
eBook
$35.99 $24.99
Print
$43.99
Subscription
$15.99 Monthly

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now
Table of content icon View table of contents Preview book icon Preview Book

Monitoring Docker

Chapter 1. Introduction to Docker Monitoring

Docker has been a recent but very important addition to a SysAdmins toolbox.

Docker describes itself as an open platform for building, shipping, and running distributed applications. This means that developers can bundle their code and pass it to their operations team. From here, they can deploy safe in the knowledge that it will be done so in a way that introduces consistency with the environment in which the code is running.

When this process is followed, it should make the age-old developers versus operations argument of "it worked on my local development server"—a thing of the past. Since before its "production ready" 1.0 release back in June 2014, there had been over 10,000 Dockerized applications available. By the end of 2014, that number had risen to over 71,000. You can see how Docker grew in 2014 by looking at the infographic that was published by Docker in early 2015, which can be found at https://blog.docker.com/2015/01/docker-project-2014-a-whirlwind-year-in-review/.

While the debate is still raging about how production ready the technology is, Docker has gained an impressive list of technology partners, including RedHat, Canonical, HP, and even Microsoft.

Companies such as Google, Spotify, Soundcloud, and CenturyLink, have all open sourced tools that support Docker in some way, shape, or form and there has also been numerous independent developers who have released apps that provide additional functionality to the core Docker product set. Also, all the companies have sprung up around the Docker ecosystem.

This book assumes that you have had some level of experience building, running, and managing Docker containers, and that you would now like to start to metrics from your running applications to further tune them, or that you would like to know when a problem occurs with a container so that you can debug any ongoing issues.

If you have never used Docker before, you may want to try one of the excellent books that serve and introduce you to all the things that Docker provides, books such as Learning Docker, Packt Publishing, or Docker's own introduction to containers, which can be found at their documentation pages, as follows:

Now, we have a brought ourselves up to speed with what Docker is; the rest of this chapter will cover the following topics:

  • How different is it to monitor containers versus more traditional servers such as virtual machines, bare metal machine, and cloud instances (Pets, Cattle, Chickens, and Snowflakes).

  • What are the minimum versions of Docker you should be running?

  • How to follow instructions on bringing up an environment locally using Vagrant in order to follow the practical exercises in this book

Pets, Cattle, Chickens, and Snowflakes


Before we start discussing the various ways in which you can monitor your containers, we should get an understanding of what a SysAdmins world looks like these days and also where containers fit into it.

A typical SysAdmin will probably be looking after an estate of servers that are hosted in either an on-site or third-party data center, some may even manage instances hosted in a public cloud such as Amazon Web Services or Microsoft Azure, and some SysAdmins may juggle all their server estates across multiple hosting environments.

Each of these different environments has its own way of doing things, as well as performing best practices. Back in February 2012, Randy Bias gave a talk at Cloudscaling that discussed architectures for open and scalable clouds. Towards the end of the slide deck, Randy introduced the concept of Pets versus Cattle (which he attributes to Bill Baker, who was then an engineer at Microsoft).

You can view the original slide deck at http://www.slideshare.net/randybias/architectures-for-open-and-scalable-clouds.

Pets versus Cattle is now widely accepted as a good analogy to describe modern hosting practices.

Pets

Pets are akin to traditional physical servers or virtual machines, as follows:

  • Each pet has a name; for example, myserver.domain.com.

  • When they're not well, you take them to the vet to help them get better. You employ SysAdmins to look after them.

  • You pay close attention to them, sometimes for years. You take backups, patch them, and ensure that they are fully documented.

Cattle

Cattle, on the other hand, represent more modern cloud computing instances, as follows:

  • You've got too many to name, so you give them numbers; for example, the URL could look something like ip123123123123.eu.public-cloud.com.

  • When they get sick, you shoot them and if your herd requires it, you replace anything you've killed: A server crashes or shows signs that it is having problems, you terminate it and your configuration automatically replaces it with an exact replica.

  • You put them in a field and watch them from far and you don't expect them to live long. Rather than monitoring the individual instances, you monitor the cluster. When more resources are needed, you add more instances and once the resource is no longer required, you terminate the instances to get you back to your base configuration.

Chickens

Next up is a term that is a good way of describing how containers fit into the Pets versus Cattle world; in a blog post title "Cloud Computing: Pets, Cattle and ... Chickens?" on ActiveState, Bernard Golden describes containers as Chickens:

  • They're more efficient than cattle when it comes to resource use. A container can boot in seconds where a instance or server can take minutes; it also uses less CPU power than a typical virtual machine or cloud instance.

  • There are many more chickens than cattle. You can quite densely pack containers onto your instances or servers.

  • Chickens tend to have a shorter lifespan than cattle and pets. Containers lend themselves to running micros-services; these containers may only be active for a few minutes.

The original blog post can be found at http://www.activestate.com/blog/2015/02/cloud-computing-pets-cattle-and-chickens.

Snowflakes

The final term is not animal-related and it describes a type of server that you defiantly don't want to have in your server estate, a Snowflake. This term was penned by Martin Fowler in a blog post titled "SnowflakeServer". Snowflakes is a term applied to "legacy" or "inherited" servers:

  • Snowflakes are delicate and are treated with kid gloves. Typically, the server has been in the data center since you started. No one knows who originally configured it and there is no documentation of it; all you know is that it is important.

  • Each one is unique and is impossible to exactly reproduce. Even the most hardened SysAdmin fears to reboot the machine incase it doesn't boot afterwards, as it is running end-of-life software that can not easily be reinstalled.

Martin's post can be found at http://martinfowler.com/bliki/SnowflakeServer.html.

So what does this all mean?

Depending on your requirements and the application you want to deploy, your containers can be launched onto either pet or cattle style servers. You can also create a clutch of chickens and have your containers run micro-services.

Also, in theory, you can replace your feared snowflake servers with a container-based application that meets all the end-of-life software requirements while remaining deployable on a modern supportable platform.

Each of the different styles of server has different monitoring requirements, in the final chapter we will look at Pets, Cattle, Chickens, and Snowflakes again and discuss the tools we have covered in the coming chapters. We will also cover best practices you should take into consideration when planning your monitoring.

Docker


While Docker hit its version 1.0 milestone over a year ago, it is still in it's infancy; with each new release comes new features, bug fixes, and even support for some early functionality that is being depreciated.

Docker itself is now a collection of several smaller projects; these include the following:

  • Docker Engine

  • Docker Machine

  • Docker Compose

  • Docker Swarm

  • Docker Hub

  • Docker Registry

  • Kitmatic

In this book, we will be using Docker Engine, Docker Compose, and the Docker Hub.

Docker Engine is the core component of the Docker project and it provides the main bulk of the Docker functionality. Whenever Docker or the docker command is mentioned in this book, I will be referring to Docker Engine.

The book assumes you have Docker Engine version 1.71 or later installed; older versions of Docker Engine may not contain the necessary functionality required to run the commands and software covered in the upcoming chapters.

Docker Compose started its life as a third-party orchestration tool called Fig before being purchased by Docker in 2014. It is described as a way of defining a multi-container application using YAML (http://yaml.org). Simply put, this means that you quickly deploy complex applications using a single command that calls a human readable configuration file.

We assume that you have Docker Compose 1.3.3 or later installed; the docker-compose.yml files mentioned in this book have been written with this version in mind.

Finally, the majority of the images we will be deploying during this book will be sourced from the Docker Hub (https://hub.docker.com/), which not only houses a public registry containing over 40,000 public images but also 100 official images. The following screenshot shows the official repositories listing on the Docker Hub website:

You can also sign up and use the Docker Hub to host your own public and private images.

Launching a local environment


Wherever possible, I will try to ensure that the practical exercises in this book will be able to be run on a local machine such as your desktop or laptop. For the purposes of this book, I will assume that your local machine is running either a recent version OS X or an up-to-date Linux distribution and has a high enough specification to run the software mentioned in this chapter.

The two tools we will be using to launch our Docker instances will also run on Windows; therefore, it should be possible to follow the instructions within this, although you may have to refer the usage guides for any changes to the syntax.

Due to the way in which Docker is architected, a lot of the content of this book will have you running commands and interacting with the command line on the virtual server that is acting as the host machine, rather than the containers themselves. Because of this, we will not be using either Docker Machine or Kitematic.

Both of these are tools provided by Docker to quickly bootstrap a Docker-enabled virtual server on your local machine, as unfortunately the host machines deployed by these tools contain a stripped down operating system that is optimized for running Docker with the smallest footprint as possible.

As we will be installing additional packages on the host machines, a stripped down "Docker only" operating system may not have the components available to meet the prerequisites of the software that we will be running in the later chapters; therefore, to ensure that there are no problems further on, we be running a full operating system.

Personally, I prefer a RPM-based operating system such as RedHat Enterprise Linux, Fedora, or CentOS, as I have been using them pretty much since the day I first logged into a Linux server.

However, as a lot of readers will be familiar with the Debian-based Ubuntu, I will be providing practical examples for both operating systems.

To ensure the experience is as consistent as possible, we will be installing Vagrant and VirtualBox to run the virtual machine that will act as a host to run our containers.

Vagrant, written by Mitchell Hashimoto, is a command line tool for creating and configuring reproducible and portable virtual machine environments. There have been numerous blog posts and articles that actually pitch Docker against Vagrant; however, in our case, the two technologies work quite well together in providing a repeatable and consistent environment.

Vagrant is available for Linux, OS X, and Windows. For details on how to install, go to the Vagrant website at https://www.vagrantup.com/.

VirtualBox is a great all round open source virtualization platform originally developed by Sun and now maintained by Oracle. It allows you to run both 32-bit and 64-bit guest operating systems on your local machine. Details on how to download and install VirtualBox can be found at https://www.virtualbox.org/; again, VirtualBox can be installed on Linux, OS X, and Windows.

Cloning the environment


The source for the environment along with the practical examples can be found on GitHub in the Monitoring Docker repository at https://github.com/russmckendrick/monitoring-docker.

To clone the repository on a terminal on your local machine, run the following commands (replacing the file path as needed):

mkdir ~/Documents/Projects
cd ~/Documents/Projects/
git clone https://github.com/russmckendrick/monitoring-docker.git

Once cloned, you should see a directory called monitoring-docker and then enter that directory, as follows:

cd ~/Documents/Projects/monitoring-docker

Running a virtual server


In the repository, you will find two folders containing the necessary Vagrant file to launch either a CentOS 7 or a Ubuntu 14.04 virtual server.

If you would like to use the CentOS 7 vagrant box, change the directory to vagrant-centos:

cd vagrant-centos

Once you are in the vagrant-centos directory, you will see that there is a Vagrant file; this file is all you need to launch a CentOS 7 virtual server. After the virtual server has been booted, the latest version of docker and docker-compose will be installed and the monitoring-docker directory will also be mounted inside the virtual machine using the mount point /monitoring-docker.

To launch the virtual server, simply type the following command:

vagrant up

This will download the latest version of the vagrant box from https://atlas.hashicorp.com/russmckendrick/boxes/centos71 and then boot the virtual server; it's a 450 MB download so it may take several minutes to download; it only has to do this once.

If all goes well, you should see something similar to the following output:

Now that you have booted the virtual server, you can connect to it using the following command:

vagrant ssh

Once logged in, you should verify that docker and docker-compose are both available:

Finally, you can try running the hello-world container using the following command:

docker run hello-world

If everything goes as expected, you should see the following output:

To try something more ambitious, you can run an Ubuntu container with the following command:

docker run -it ubuntu bash

Before we launch and enter the Ubuntu container, lets confirm that we are running the CentOS host machine by checking the release file that can be found in /etc:

Now, we can launch the Ubuntu container. Using the same command, we can confirm that we are inside the Ubuntu container by viewing its release file:

To exit the container just type in exit. This will stop the container from running, as it has terminated the only running process within the container, which was bash, and returned you to the host CentOS machine.

As you can see here from our CentOS 7 host, we have launched and removed an Ubuntu container.

Both the CentOS 7 and Ubuntu Vagrant files will configure a static IP address on your virtual machine. It is 192.168.33.10; also, there is a DNS record for this IP address available at docker.media-glass.es. These will allow you to access any containers that expose themselves to a browser at either http://192.168.33.10/ or http://docker.media-glass.es/.

Tip

The URL http://docker.media-glass.es/ will only work while the vagrant box is up, and you have a container running which serves Web pages.

You can see this in action by running the following command:

docker run -d -p 80:80russmckendrick/nginx-php

Tip

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

This will download and launch a container running NGINX. You can then go to http://192.168.33.10/ or http://docker.media-glass.es/ in your browser; you should see a forbidden page. This is because we have not yet given NGINX any content to serve (more on this will be covered later in the book):

For more examples and ideas, go to the website at http://docs.docker.com/userguide/.

Halting the virtual server


To log out of the virtual server and return to your local machine, you type exit.

You should now see your local machine's terminal prompt; however, the virtual server you booted will still be running in the background happily, using resources, until you either power it down using the following command:

vagrant halt

Terminate the virtual server altogether using vagrant destroy:

vagrant destroy

To check the current status of the virtual server, you can run the following command:

vagrant status

The result of the preceding command is given in the following output:

Either powering the virtual server back on or creating it from scratch again, can be achieved by issuing the vagrant up command again.

The preceding details show how to use the CentOS 7 vagrant box. If you would prefer to launch an Ubuntu 14.04 virtual server, you can download and install the vagrant box by going into the vagrant-ubuntu directory using the following command:

cd ~/Documents/Projects/monitoring-docker/vagrant-ubuntu
vagrant up

From here, you will be able run vagrant up and follow the same instructions used to boot and interact with the CentOS 7 virtual server.

Summary


In this chapter, we talked about different types of server and also discussed how your containerized applications can fit into each of the categories. We have also installed VirtualBox and used Vagrant to launch either a CentOS 7 or Ubuntu 14.04 virtual server, with docker and docker-compose installed.

Our new virtual server environment will be used throughout the upcoming chapters to test the various different types of monitoring. In the next chapter, we will start our journey by using Docker's in-built functionality to explore metrics about our running containers.

Left arrow icon Right arrow icon

Key benefits

  • • Get the only book that covers multiple methods to monitoring Docker Containers -your one-stop solution to all your Docker monitoring needs
  • • Learn how to implement third-party tools such as Datalog, cAdvisor, and more with this example-rich, hands-on guide
  • • Learn how to efficiently monitor Docker containers and get an edge over other administrators while maintaining apps on these containers

Description

This book will show you how monitoring containers and keeping a keen eye on the working of applications helps improve the overall performance of the applications that run on Docker. With the increased adoption of Docker containers, the need to monitor which containers are running, what resources they are consuming, and how these factors affect the overall performance of the system has become the need of the moment. This book covers monitoring containers using Docker's native monitoring functions, various plugins, as well as third-party tools that help in monitoring. Well start with how to obtain detailed stats for active containers, resources consumed, and container behavior. We also show you how to use these stats to improve the overall performance of the system. Next, you will learn how to use SysDig to both view your containers performance metrics in real time and record sessions to query later. By the end of this book, you will have a complete knowledge of how to implement monitoring for your containerized applications and make the most of the metrics you are collecting

What you will learn

• Discover the tools built into Docker to gain an insight into your containers’ performance • Augment Docker’s built-in tools with modern tools such as cAdvisor from Google, SysDig by Draios, and Soundcloud’s Prometheus • Integrate the monitoring of your containers with more traditional monitoring solutions such as Zabbix • Take advantage of the various SaaS offerings from third parties to move monitoring away from your local infrastructure and into the cloud • Discover the various ways to ship your applications’ logs from the container to a central logging service • Get the most out of your application and resources with the right implementation of your monitoring method

Product Details

Country selected

Publication date : Dec 15, 2015
Length 148 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781785882753
Vendor :
Docker
Languages :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now

Product Details


Publication date : Dec 15, 2015
Length 148 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781785882753
Vendor :
Docker
Languages :

Table of Contents

15 Chapters
Monitoring Docker Chevron down icon Chevron up icon
Credits Chevron down icon Chevron up icon
About the Author Chevron down icon Chevron up icon
About the Reviewer Chevron down icon Chevron up icon
www.PacktPub.com Chevron down icon Chevron up icon
Preface Chevron down icon Chevron up icon
1. Introduction to Docker Monitoring Chevron down icon Chevron up icon
2. Using the Built-in Tools Chevron down icon Chevron up icon
3. Advanced Container Resource Analysis Chevron down icon Chevron up icon
4. A Traditional Approach to Monitoring Containers Chevron down icon Chevron up icon
5. Querying with Sysdig Chevron down icon Chevron up icon
6. Exploring Third Party Options Chevron down icon Chevron up icon
7. Collecting Application Logs from within the Container Chevron down icon Chevron up icon
8. What Are the Next Steps? Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Empty star icon Empty star icon Empty star icon Empty star icon Empty star icon 0
(0 Ratings)
5 star 0%
4 star 0%
3 star 0%
2 star 0%
1 star 0%
Top Reviews
No reviews found
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.