Home Cloud & Networking Docker High Performance - Second Edition

Docker High Performance - Second Edition

By Allan Espinosa , Russ McKendrick
books-svg-icon Book
eBook $19.99 $13.98
Print $26.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $19.99 $13.98
Print $26.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
About this book
Docker is an enterprise-grade container platform that allows you to build and deploy your apps. Its portable format lets you run your code right from your desktop workstations to popular cloud computing providers. This comprehensive guide will improve your Docker work?ows and ensure your application's production environment runs smoothly. This book starts with a refresher on setting up and running Docker and details the basic setup for creating a Docker Swarm cluster. You will then learn how to automate this cluster by using the Chef server and cookbooks. After that, you will run the Docker monitoring system with Prometheus and Grafana, and deploy the ELK stack. You will also learn best practices for optimizing Docker images. After deploying containers with the help of Jenkins, you will then move on to a tutorial on using Apache JMeter to analyze your application's performance. You will learn how to use Docker Swarm and NGINX to load-balance your application, and how common debugging tools in Linux can be used to troubleshoot Docker containers. By the end of this book, you will be able to integrate all the optimizations that you have learned and put everything into practice in your applications.
Publication date:
April 2019
Publisher
Packt
Pages
174
ISBN
9781789807219

 

Chapter 1. Preparing Docker Hosts

Docker allows us to deliver applications to our customers faster. It simplifies the workflows needed to get code from development to production by being able to easily create and launch Docker containers. This chapter will be a quick refresher on how to get our environment ready to run Docker-based development and operations workflow by doing the following:

  • Preparing a Docker host
  • Enabling remote access to Docker hosts
  • Building a Docker Swarm cluster

Most of the parts of this chapter are concepts that we are already familiar with and these  are readily available in the Docker documentation website. This chapter shows selected commands and interactions with the Docker host that will be used in the succeeding chapters.

 

Preparing a Docker host


It is assumed that we are already familiar on how to set up a Docker host. For most of the chapters of this book, we will run against the following environment unless mentioned explicitly:

  • Operating System: CentOS 7.5
  • Docker version: 18.09.0

The following commands display the operating system and Docker Engine version running inside our Docker host:

$ ssh dockerhost
dockerhost$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID:    CentOS
Description: CentOS Linux release 7.5.1804 (Core)
Release:    7.5.1804
Codename:   Core
dockerhost$ docker version
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:48:22 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:19:08 2018
  OS/Arch:          linux/amd64
  Experimental:     false

Note

If we haven't set up our Docker environment, we can follow the instructions in the Docker website found in https://docs.docker.com/install/linux/docker-ce/centos/ to prepare our Docker host.

 

Enabling remote access


Instead of logging in remotely to our Docker host to run containers, we will access the Docker host by enabling the remote API in Docker Engine. This allows us to manage our Docker containers from our client workstation or continuous delivery server. We will then interact with our Docker host to represent our production environment. The remote API will then be used from our client workstation to perform deployments of our Docker containers.

This section will cover the steps to secure and enable remote access to our Docker host:

  • Setting up a certificate authority
  • Reconfiguring Docker Engine to enable remote access
  • Configuring the Docker client for remote access

Setting up a certificate authority

For the rest of this section, we will be installing TLS certificates in both our Docker host server and client. To ensure trusted communication between the server and client, we will be setting up a Public Key Infrastructure (PKI). This will allow both the Docker engine running in our host and our Docker client to make the connection to verify the identity of each other. 

The first step in building the PKI is setting up the Certificate Authority (CA). A CA is a trusted third party that issues digital certificates to members of our PKI, namely our Docker host and client.

In the next few steps, we will set up our CA inside of our client workstation:

  1. First, we will create a directory for our PKI and generate the CA's private key in a file called ca-key.pem:
client$ mkdir ~/ca
client$ cd ~/ca
client$ openssl genrsa -aes256 -out ca-key.pem 4096

Generating RSA private key, 4096 bit long modulus
......................................++
................................................++
e is 65537 (0x10001)

Enter pass phrase for ca-key.pem: ****
Verifying - Enter pass phrase for ca-key.pem: ****

We need to remember the passphrases set for our CA's private key, as we will always need them for the rest of this chapter.

  1. Next, we make sure that this private key is secure by restricting read and write access to us:
client$ chmod 600 ca-key.pem
client$ ls -l ca-key.pem
-rw-------. 1 dockeruser group 3326 Dec  2 20:45 ca-key.pem
  1. Finally, we will generate a certificate for our CA that is self-signed. Let's type the following command to place the self-signed certificate in a file called ca.pem:
client$ openssl req -key ca-key.pem -new -x509  \
-subj '/CN=Certificate Authority' \
-sha256 -days 365 -out ca.pem
Enter pass phrase for ca-key.pem: ****

We now have a CA, and we will be distributing its ca.pemcertificate to our Docker host and client later in this section. For now, let's inspect the generate certificate:

client$ cat ca.pem

-----BEGIN CERTIFICATE-----
      MIIFEzCCAvugAwIBAgIJAM19ce5sap+kMA0GCSqGSIb3DQEBCwUAMCAxHjAcBgNV
      BAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xODEyMDYwMzQ5MTNaFw0xOTEy
      MDYwMzQ5MTNaMCAxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTCCAiIw
      DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAKCESs7QpRZ78v8p2nKomCGABqCN
      b3E0vBpjveTnjA4kOEWVsHloq2o66yuuNff75GNWghzq791KyKJOy/dehNL9DauA
      DD3DJh0+uaOGn547W827Z37wJ64acNyvIQjyiyeLrpF4BzzxaZ/AJFVgqar5Kuqc
      qiOG3GUYcnfu6mpmlKoa1XqBtSQ+A2fd4/mpXC0zrDrz9MSEOCs5/Xm6/faexYae
      V8gBkCYWiUVUi+RRRc2vU1LzuiI5FsXmD3kNHCjNIbYIoyqKMzbTJjEffhN+5B/V
      Rc3qfRmfoEv8P0Hc4Wx55qH8BLWwhvNFAZ+nre+j7zPz+dTfLVyveOPxErHaI1V8
      WH9qEVf+haNqUBrjNCuL+xyVNx7evPygD88jyZDWLK5Y0JTh2GSPqMeVi3hSKzNP
      GbVjT8tmkCUEsYbSJg2vkPYJR4aC8LLdJsjr7wkWBF1IcYYZpLo3EsUnkjNi7MGS
      pGdLob3UToekXaA1D6esDhlEB+3Tt/RWJkS91ijUiDs2kTSmDfnxUQGyeD4wx/rj
      lPFRSLdUUYiFcdI5VegZVSqYxW/Qw2/t+GvoLkrOrggqY1f++XUgK5hSoT8EqgiG
      SjapkgphMEquVP8UlZ3jC0VmgwFnRUEdqau6yLWMYG6TvLkyVi1Vmfam7CoB1aDn
      TccUszk+rezX+1nJAgMBAAGjUDBOMB0GA1UdDgQWBBTrqfPKO0i2peZ6Hd/BYOMq
      WXD9kDAfBgNVHSMEGDAWgBTrqfPKO0i2peZ6Hd/BYOMqWXD9kDAMBgNVHRMEBTAD
      AQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCa6SPGncEZSWu0WLfkh1mERa9JfBQzJFpv
      1E7M3tZeFyJS7LfXdcf9WEAaWqTpha87A+5g9uBi/whYk47dyTik07/k+CyF112i
      9GXK8j/UNCjAMOSluOCxpIsmMXp2Dn+ma21msN1K/lHK0ZhGWB9ZDggvdzRRPjic
      Dq3aQ49ATHQHGg9cqgZO0zXtcQYaHfCNds5YLNVL66eDhuN91V2MEqWtRDHfr0vA
      F3KldXfQ/clnrjGLqo7a3oR1R4QofQ03bV+PRIgub+l3Fee1D68BqF9dLRjUABd2
      zm5OzNAmmHPSGWGvOxylvPrUS0ulUzMUWdoXN85SDdLHFXTXwpbD/GgqK+Y3BTgO
      7d+mOoTHVEdw2gUXLaqeEchBge2Kh/LQtiN7Zp8OY7snX66Z8tF6W2MKhnSpDzcW
      J4WMbmaRqsTEeaRk0aTWkhBZukSZf4zjaa/abF+iRvU5c1OGS9GmYfuGq3Tlj+Xo
      JZNuKp9HzOPaj8qiD0DJW9EnuZ24zzpDSiSdmOdARcaaFFKhW8i+SVP6VqrAR3Nb
      OL8ne6w6kdoiq4+hPKfWVS9Yh0aQstJMNP91Nnw3J+aRz9eN03jpl/z18vHhW/xl
      nYJrB2KlC7SOnUT7TMJr4O5Aw1SidxMH6NLiiC1jbTWXDMuYL8UghDIk9Ne/WhBd
qg0sW+boLw==
-----END CERTIFICATE-----

Enabling remote access in Docker Engine

Now that we have a CA for our PKI, we can use this CA to verify the identity of our Docker host. The following steps will prepare the identity of our Docker host:

  1. First, we log in to our Docker host. In here, we will generate a private key that will secure the remote API being served by running Docker Engine. The following command will save the private key into a file called /etc/docker/server-key.pem:
dockerhost$ openssl genrsa -out /etc/docker/server-key.pem 2048

Generating RSA private key, 2048 bit long modulus
................................+++
........+++
e is 65537 (0x10001)
  1. Next, we make sure that this file is secure and can only be accessed by the Docker Engine daemon (through the root user):
dockerhost$ chmod 600 /etc/docker/server-key.pem
dockerhost$ ls -l /etc/docker/server-key.pem

-rw-------. 1 root root 1675 Dec  2 21:09 /etc/docker/server-key.pem
  1. Now that the private key is ready, we will use this file to generate a Certificate Signing Request(CSR). openssl req following command will generate a CSR:
dockerhost$ openssl req -key /etc/docker/server-key.pem 
-new -subj "/CN=dockerhost" -sha256 -out           
dockerhost.csr
dockerhost$ ls -l dockerhost.csr
-rw-r--r--. 1 root root 891 Dec  2 21:33 dockerhost.csr
  1. Next, we go back to our client workstation where our CA's files are hosted. In here, we will download the CSR from our Docker host:
client$ scp dockerhost:~/dockerhost.csr dockerhost.csr
dockerhost.csr           100%  891     1.5MB/s   00:00
  1. We now prepare an OpenSSL configuration server-ext.cnffile that indicates that the certificates our CA will issue are used for server authentication:
extendedKeyUsage = serverAuth
  1. Finally, we can sign the CSR with our CA. The following command will place our Docker host's signed certificate in a file called dockerhost.pem:
client$ cd ~/ca
client$ openssl x509 -req -CA ca.pem -CAkey ca-key.pem \
-CAcreateserial -extfile server-ext.cnf \
-in dockerhost.csr -out dockerhost.pem

Signature ok
subject=/CN=dockerhost
Getting CA Private Key
Enter pass phrase for ca-key.pem: ****

Now that we have the identity of our Docker host verified by our CA, we can now enable the secure TCP port in our Docker host. We will bring up the secure remote API with the following steps:

  1. Let's now go back into our Docker host. Here, we will copy the certificates of our Docker host and CA from our client workstation:
dockerhost$ scp client:~/ca/ca.pem /etc/docker/ca.pem
ca.pem                  100% 1911     1.1MB/s   00:00
dockerhost$ scp client:~/ca/dockerhost.pem /etc/docker/server.pem
dockerhost.pem                      100% 1428     1.2MB/s   00:00
  1. Now that our TLS assets are in place, let's now reconfigure the Docker Engine daemon file, /etc/docker/daemon.json, to use those certificates:
{
  "tlsverify": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlskey": "/etc/docker/server-key.pem",
  "tlscert": "/etc/docker/server.pem"
}
  1. Next, we configure the Docker Engine daemon to listen to a secure port by creating a systemd override file /etc/systemd/system/docker.service.d/override.conf:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:// -H tcp://0.0.0.0:2376
  1. Finally, we are now ready to restart Docker Engine:
dockerhost$ systemctl daemon-reload
dockerhost$ systemctl restart docker.service

Our Docker host is now ready and serving a secure API.

Connecting remotely from the Docker client

Now that our Docker host is secure, it won't respond to requests from our Docker client yet. The Docker host will only respond to requests being made by clients verified by our CA.

The following steps will generate an identity for our Docker client:

  1. First, let's generate the private key of our Docker client in ~/.docker/key.pem:
client$ openssl genrsa -out ~/.docker/key.pem 4096

Generating RSA private key, 4096 bit long modulus
...................++
      .........................................................++
e is 65537 (0x10001)
  1. Next, we make sure that this private key is restricted to us for viewing:
client$ chmod 600 ~/.docker/key.pem
  1. We now generate the CSR for client in a file called client.csr:
client$ openssl req -subj '/CN=client' -new \
-key ~/.docker/key.pem -out client.csr
  1. Now that our CSR is ready, we will now create an OpenSSL configuration to indicate that certificates will be used for client authentication. The OpenSSL command following creates this configuration in a file called ~/ca/client-ext.cnf:
extendedKeyUsage = clientAuth
  1. Finally, we are ready to issue the certificate for our Docker client. The following command writes our Docker client's certificate to ~/.docker/cert.pem:
 client$ openssl x509 -req -CA ca.pem 
      -CAkey ca-key.pem -CAcreateserial 
      -extfile client-ext.cnf -in       
      ~/client.csr -out ~/.docker/cert.pem
      Signature ok
      subject=/CN=client
      Getting CA Private Key
      Enter pass phrase for ca-key.pem: ****
  1. To complete our client's TLS configuration, we will also deploy our CA's certificate in our ~/.docker directory file:
      client$ cp ca.pem ~/.docker/ca.pem
  1. Finally, we indicate to our Docker client that we will be connecting securely to our remote Docker host by exporting the following environment variables:
      client$ export DOCKER_HOST=tcp://dockerhost:2376
      client$ export DOCKER_TLS_VERIFY=true

Congratulations! We now have a secure communication channel between our Docker client and Docker host. To verify the connection, we can run the following command and show information about our remote Docker host:

client$ docker info
 

Building a Docker Swarm cluster


Docker introduced swarm mode to its Docker engine from version 1.12.0. Docker Swarm allows us to pool together multiple Docker hosts to deploy our containers in a scalable and high availability way. In this section, we will build a small Docker Swarm cluster.

Let's dive into building our cluster with the following steps:

  1. First, we will go to our Docker host and initialize it as a managermanager is responsible for maintaining the state of our Docker Swarm cluster. It also dispatches tasks to other Docker hosts in our cluster. Let's type the following command to begin the initialization:
dockerhost$ docker swarm init
Swarm initialized: current node (w49smc2ciy100gaecgx77yir3) 
is now a manager

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-4wbs...aq2r \
172.16.132.187:2377

The preceding command generated a token that will be used by other Docker hosts to join our cluster.

  1. Next, we will go to a new Docker host called node1. We use the token from the previous step to make this Docker host join our Docker Swarm cluster as a worker. Workers are members of the cluster that are responsible for running our containers. Let's now type the following command to make this new node join our cluster:
node1$ docker swarm join --token SWMTKN-1-4...aq2r \ 172.16.132.187:2377
This node joined a swarm as a worker.

Note

We can scale out our Docker Swarm cluster by adding more managers and workers using the same Docker Swarm join command. More details can be found in the upstream Docker documentation at https://docs.docker.com/engine/swarm/join-nodes.

We have now finished setting up our Docker Swarm cluster. Let's go back to our Docker client workstation and confirm the members of our cluster:

client$ docker node ls
ID       HOSTNAME   STATUS  AVAILABILITY MANAGER STATUS ENGINE VERSION
w49smc * dockerhost Ready   Active       Leader         18.09.0
2e0aif   node1      Ready   Active                      18.09.0
 

Summary


Hopefully, at this point, we have become familiar with interacting with Docker Engine. We have prepared a Docker host where we will run our containers. We built a PKI to ensure secure communication between our Docker host and our Docker client workstation, and we built a small Docker Swarm cluster consisting of multiple Docker hosts.

We accomplished all of this by logging in to servers and manually typing configuration commands. In the next chapter, we'll learn to automate provisioning these Docker hosts—and save us from typing!

About the Authors
  • Allan Espinosa

    Allan Espinosa is a DevOps practitioner and an active open source contributor to various distributed system tools, such as Docker and Chef. Allan maintains several Docker images for popular open source software that were popular even before their official release from the upstream open source groups. Throughout his career, Allan has worked on large distributed systems containing hundreds to thousands of servers in production. He has built scalable applications on various platforms, ranging from large supercomputing centers to production clusters in the enterprise. He is currently managing distributed systems at scale for Bloomberg, where he oversees the company's Hadoop infrastructure. Allan can be contacted through his Twitter handle, @AllanEspinosa.

    Browse publications by this author
  • Russ McKendrick

    Russ McKendrick is an experienced DevOps practitioner and system administrator with a passion for automation and containers. He has been working in IT and related industries for the better part of 30 years. During his career, he has had varied responsibilities in many different sectors, including first-line, second-line, and senior support in both client-facing and internal teams for small and large organizations. He works almost exclusively with Linux, using open-source systems and tools across dedicated hardware and virtual machines hosted in public and private clouds at Node4, where he holds the title of practice manager (SRE and DevOps). He also buys way too many records!

    Browse publications by this author
Latest Reviews (1 reviews total)
Site badly designed. The basket does not work under Firefox/Mac and downloads of colour PDFs not working.
Docker High Performance - Second Edition
Unlock this book and the full library FREE for 7 days
Start now