Docker High Performance - Second Edition

By Allan Espinosa , Russ McKendrick
    What do you get with a Packt Subscription?

  • Instant access to this title and 7,500+ eBooks & Videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
About this book
Docker is an enterprise-grade container platform that allows you to build and deploy your apps. Its portable format lets you run your code right from your desktop workstations to popular cloud computing providers. This comprehensive guide will improve your Docker work?ows and ensure your application's production environment runs smoothly. This book starts with a refresher on setting up and running Docker and details the basic setup for creating a Docker Swarm cluster. You will then learn how to automate this cluster by using the Chef server and cookbooks. After that, you will run the Docker monitoring system with Prometheus and Grafana, and deploy the ELK stack. You will also learn best practices for optimizing Docker images. After deploying containers with the help of Jenkins, you will then move on to a tutorial on using Apache JMeter to analyze your application's performance. You will learn how to use Docker Swarm and NGINX to load-balance your application, and how common debugging tools in Linux can be used to troubleshoot Docker containers. By the end of this book, you will be able to integrate all the optimizations that you have learned and put everything into practice in your applications.
Publication date:
April 2019


Chapter 1. Preparing Docker Hosts

Docker allows us to deliver applications to our customers faster. It simplifies the workflows needed to get code from development to production by being able to easily create and launch Docker containers. This chapter will be a quick refresher on how to get our environment ready to run Docker-based development and operations workflow by doing the following:

  • Preparing a Docker host
  • Enabling remote access to Docker hosts
  • Building a Docker Swarm cluster

Most of the parts of this chapter are concepts that we are already familiar with and these  are readily available in the Docker documentation website. This chapter shows selected commands and interactions with the Docker host that will be used in the succeeding chapters.


Preparing a Docker host

It is assumed that we are already familiar on how to set up a Docker host. For most of the chapters of this book, we will run against the following environment unless mentioned explicitly:

  • Operating System: CentOS 7.5
  • Docker version: 18.09.0

The following commands display the operating system and Docker Engine version running inside our Docker host:

$ ssh dockerhost
dockerhost$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID:    CentOS
Description: CentOS Linux release 7.5.1804 (Core)
Release:    7.5.1804
Codename:   Core
dockerhost$ docker version
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:48:22 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:19:08 2018
  OS/Arch:          linux/amd64
  Experimental:     false


If we haven't set up our Docker environment, we can follow the instructions in the Docker website found in to prepare our Docker host.


Enabling remote access

Instead of logging in remotely to our Docker host to run containers, we will access the Docker host by enabling the remote API in Docker Engine. This allows us to manage our Docker containers from our client workstation or continuous delivery server. We will then interact with our Docker host to represent our production environment. The remote API will then be used from our client workstation to perform deployments of our Docker containers.

This section will cover the steps to secure and enable remote access to our Docker host:

  • Setting up a certificate authority
  • Reconfiguring Docker Engine to enable remote access
  • Configuring the Docker client for remote access

Setting up a certificate authority

For the rest of this section, we will be installing TLS certificates in both our Docker host server and client. To ensure trusted communication between the server and client, we will be setting up a Public Key Infrastructure (PKI). This will allow both the Docker engine running in our host and our Docker client to make the connection to verify the identity of each other. 

The first step in building the PKI is setting up the Certificate Authority (CA). A CA is a trusted third party that issues digital certificates to members of our PKI, namely our Docker host and client.

In the next few steps, we will set up our CA inside of our client workstation:

  1. First, we will create a directory for our PKI and generate the CA's private key in a file called ca-key.pem:
client$ mkdir ~/ca
client$ cd ~/ca
client$ openssl genrsa -aes256 -out ca-key.pem 4096

Generating RSA private key, 4096 bit long modulus
e is 65537 (0x10001)

Enter pass phrase for ca-key.pem: ****
Verifying - Enter pass phrase for ca-key.pem: ****

We need to remember the passphrases set for our CA's private key, as we will always need them for the rest of this chapter.

  1. Next, we make sure that this private key is secure by restricting read and write access to us:
client$ chmod 600 ca-key.pem
client$ ls -l ca-key.pem
-rw-------. 1 dockeruser group 3326 Dec  2 20:45 ca-key.pem
  1. Finally, we will generate a certificate for our CA that is self-signed. Let's type the following command to place the self-signed certificate in a file called ca.pem:
client$ openssl req -key ca-key.pem -new -x509  \
-subj '/CN=Certificate Authority' \
-sha256 -days 365 -out ca.pem
Enter pass phrase for ca-key.pem: ****

We now have a CA, and we will be distributing its ca.pemcertificate to our Docker host and client later in this section. For now, let's inspect the generate certificate:

client$ cat ca.pem


Enabling remote access in Docker Engine

Now that we have a CA for our PKI, we can use this CA to verify the identity of our Docker host. The following steps will prepare the identity of our Docker host:

  1. First, we log in to our Docker host. In here, we will generate a private key that will secure the remote API being served by running Docker Engine. The following command will save the private key into a file called /etc/docker/server-key.pem:
dockerhost$ openssl genrsa -out /etc/docker/server-key.pem 2048

Generating RSA private key, 2048 bit long modulus
e is 65537 (0x10001)
  1. Next, we make sure that this file is secure and can only be accessed by the Docker Engine daemon (through the root user):
dockerhost$ chmod 600 /etc/docker/server-key.pem
dockerhost$ ls -l /etc/docker/server-key.pem

-rw-------. 1 root root 1675 Dec  2 21:09 /etc/docker/server-key.pem
  1. Now that the private key is ready, we will use this file to generate a Certificate Signing Request(CSR). openssl req following command will generate a CSR:
dockerhost$ openssl req -key /etc/docker/server-key.pem 
-new -subj "/CN=dockerhost" -sha256 -out           
dockerhost$ ls -l dockerhost.csr
-rw-r--r--. 1 root root 891 Dec  2 21:33 dockerhost.csr
  1. Next, we go back to our client workstation where our CA's files are hosted. In here, we will download the CSR from our Docker host:
client$ scp dockerhost:~/dockerhost.csr dockerhost.csr
dockerhost.csr           100%  891     1.5MB/s   00:00
  1. We now prepare an OpenSSL configuration server-ext.cnffile that indicates that the certificates our CA will issue are used for server authentication:
extendedKeyUsage = serverAuth
  1. Finally, we can sign the CSR with our CA. The following command will place our Docker host's signed certificate in a file called dockerhost.pem:
client$ cd ~/ca
client$ openssl x509 -req -CA ca.pem -CAkey ca-key.pem \
-CAcreateserial -extfile server-ext.cnf \
-in dockerhost.csr -out dockerhost.pem

Signature ok
Getting CA Private Key
Enter pass phrase for ca-key.pem: ****

Now that we have the identity of our Docker host verified by our CA, we can now enable the secure TCP port in our Docker host. We will bring up the secure remote API with the following steps:

  1. Let's now go back into our Docker host. Here, we will copy the certificates of our Docker host and CA from our client workstation:
dockerhost$ scp client:~/ca/ca.pem /etc/docker/ca.pem
ca.pem                  100% 1911     1.1MB/s   00:00
dockerhost$ scp client:~/ca/dockerhost.pem /etc/docker/server.pem
dockerhost.pem                      100% 1428     1.2MB/s   00:00
  1. Now that our TLS assets are in place, let's now reconfigure the Docker Engine daemon file, /etc/docker/daemon.json, to use those certificates:
  "tlsverify": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlskey": "/etc/docker/server-key.pem",
  "tlscert": "/etc/docker/server.pem"
  1. Next, we configure the Docker Engine daemon to listen to a secure port by creating a systemd override file /etc/systemd/system/docker.service.d/override.conf:
ExecStart=/usr/bin/dockerd -H unix:// -H tcp://
  1. Finally, we are now ready to restart Docker Engine:
dockerhost$ systemctl daemon-reload
dockerhost$ systemctl restart docker.service

Our Docker host is now ready and serving a secure API.

Connecting remotely from the Docker client

Now that our Docker host is secure, it won't respond to requests from our Docker client yet. The Docker host will only respond to requests being made by clients verified by our CA.

The following steps will generate an identity for our Docker client:

  1. First, let's generate the private key of our Docker client in ~/.docker/key.pem:
client$ openssl genrsa -out ~/.docker/key.pem 4096

Generating RSA private key, 4096 bit long modulus
e is 65537 (0x10001)
  1. Next, we make sure that this private key is restricted to us for viewing:
client$ chmod 600 ~/.docker/key.pem
  1. We now generate the CSR for client in a file called client.csr:
client$ openssl req -subj '/CN=client' -new \
-key ~/.docker/key.pem -out client.csr
  1. Now that our CSR is ready, we will now create an OpenSSL configuration to indicate that certificates will be used for client authentication. The OpenSSL command following creates this configuration in a file called ~/ca/client-ext.cnf:
extendedKeyUsage = clientAuth
  1. Finally, we are ready to issue the certificate for our Docker client. The following command writes our Docker client's certificate to ~/.docker/cert.pem:
 client$ openssl x509 -req -CA ca.pem 
      -CAkey ca-key.pem -CAcreateserial 
      -extfile client-ext.cnf -in       
      ~/client.csr -out ~/.docker/cert.pem
      Signature ok
      Getting CA Private Key
      Enter pass phrase for ca-key.pem: ****
  1. To complete our client's TLS configuration, we will also deploy our CA's certificate in our ~/.docker directory file:
      client$ cp ca.pem ~/.docker/ca.pem
  1. Finally, we indicate to our Docker client that we will be connecting securely to our remote Docker host by exporting the following environment variables:
      client$ export DOCKER_HOST=tcp://dockerhost:2376
      client$ export DOCKER_TLS_VERIFY=true

Congratulations! We now have a secure communication channel between our Docker client and Docker host. To verify the connection, we can run the following command and show information about our remote Docker host:

client$ docker info

Building a Docker Swarm cluster

Docker introduced swarm mode to its Docker engine from version 1.12.0. Docker Swarm allows us to pool together multiple Docker hosts to deploy our containers in a scalable and high availability way. In this section, we will build a small Docker Swarm cluster.

Let's dive into building our cluster with the following steps:

  1. First, we will go to our Docker host and initialize it as a managermanager is responsible for maintaining the state of our Docker Swarm cluster. It also dispatches tasks to other Docker hosts in our cluster. Let's type the following command to begin the initialization:
dockerhost$ docker swarm init
Swarm initialized: current node (w49smc2ciy100gaecgx77yir3) 
is now a manager

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-4wbs...aq2r \

The preceding command generated a token that will be used by other Docker hosts to join our cluster.

  1. Next, we will go to a new Docker host called node1. We use the token from the previous step to make this Docker host join our Docker Swarm cluster as a worker. Workers are members of the cluster that are responsible for running our containers. Let's now type the following command to make this new node join our cluster:
node1$ docker swarm join --token SWMTKN-1-4...aq2r \
This node joined a swarm as a worker.


We can scale out our Docker Swarm cluster by adding more managers and workers using the same Docker Swarm join command. More details can be found in the upstream Docker documentation at

We have now finished setting up our Docker Swarm cluster. Let's go back to our Docker client workstation and confirm the members of our cluster:

client$ docker node ls
w49smc * dockerhost Ready   Active       Leader         18.09.0
2e0aif   node1      Ready   Active                      18.09.0


Hopefully, at this point, we have become familiar with interacting with Docker Engine. We have prepared a Docker host where we will run our containers. We built a PKI to ensure secure communication between our Docker host and our Docker client workstation, and we built a small Docker Swarm cluster consisting of multiple Docker hosts.

We accomplished all of this by logging in to servers and manually typing configuration commands. In the next chapter, we'll learn to automate provisioning these Docker hosts—and save us from typing!

About the Authors
  • Allan Espinosa

    Allan Espinosa is a DevOps practitioner and an active open source contributor to various distributed system tools, such as Docker and Chef. Allan maintains several Docker images for popular open source software that were popular even before their official release from the upstream open source groups. Throughout his career, Allan has worked on large distributed systems containing hundreds to thousands of servers in production. He has built scalable applications on various platforms, ranging from large supercomputing centers to production clusters in the enterprise. He is currently managing distributed systems at scale for Bloomberg, where he oversees the company's Hadoop infrastructure. Allan can be contacted through his Twitter handle, @AllanEspinosa.

    Browse publications by this author
  • Russ McKendrick

    Russ McKendrick is an experienced DevOps practitioner and system administrator with a passion for automation and containers. He has been working in IT and related industries for the better part of 27 years. During his career, he has had varied responsibilities in many different sectors, including first-line, second-line, and senior support in both client-facing and internal teams for small and large organizations. He works almost exclusively with Linux, using open source systems and tools across both dedicated hardware and virtual machines hosted in public and private clouds at N4Stack, which is a Node4 company, where he holds the title of practice manager (SRE and DevOps). He also buys way too many records!

    Browse publications by this author
Docker High Performance - Second Edition
Unlock this book and the full library FREE for 7 days
Start now