Docker allows us to deliver applications to our customers faster. It simplifies the workflows needed to get code from development to production by being able to easily create and launch Docker containers. This chapter will be a quick refresher on how to get our environment ready to run Docker-based development and operations workflow by doing the following:
- Preparing a Docker host
- Enabling remote access to Docker hosts
- Building a Docker Swarm cluster
Most of the parts of this chapter are concepts that we are already familiar with and these are readily available in the Docker documentation website. This chapter shows selected commands and interactions with the Docker host that will be used in the succeeding chapters.
It is assumed that we are already familiar on how to set up a Docker host. For most of the chapters of this book, we will run against the following environment unless mentioned explicitly:
- Operating System: CentOS 7.5
- Docker version: 18.09.0
The following commands display the operating system and Docker Engine version running inside our Docker host:
$ ssh dockerhost dockerhost$ lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.5.1804 (Core) Release: 7.5.1804 Codename: Core dockerhost$ docker version Version: 18.09.0 API version: 1.39 Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:48:22 2018 OS/Arch: linux/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 18.09.0 API version: 1.39 (minimum version 1.12) Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:19:08 2018 OS/Arch: linux/amd64 Experimental: false
Note
If we haven't set up our Docker environment, we can follow the instructions in the Docker website found in https://docs.docker.com/install/linux/docker-ce/centos/ to prepare our Docker host.
Instead of logging in remotely to our Docker host to run containers, we will access the Docker host by enabling the remote API in Docker Engine. This allows us to manage our Docker containers from our client workstation or continuous delivery server. We will then interact with our Docker host to represent our production environment. The remote API will then be used from our client workstation to perform deployments of our Docker containers.
This section will cover the steps to secure and enable remote access to our Docker host:
- Setting up a certificate authority
- Reconfiguring Docker Engine to enable remote access
- Configuring the Docker client for remote access
For the rest of this section, we will be installing TLS certificates in both our Docker host server and client. To ensure trusted communication between the server and client, we will be setting up a Public Key Infrastructure (PKI). This will allow both the Docker engine running in our host and our Docker client to make the connection to verify the identity of each other.
The first step in building the PKI is setting up the Certificate Authority (CA). A CA is a trusted third party that issues digital certificates to members of our PKI, namely our Docker host and client.
In the next few steps, we will set up our CA inside of our client workstation:
- First, we will create a directory for our PKI and generate the CA's private key in a file called
ca-key.pem
:
client$ mkdir ~/ca client$ cd ~/ca client$ openssl genrsa -aes256 -out ca-key.pem 4096 Generating RSA private key, 4096 bit long modulus ......................................++ ................................................++ e is 65537 (0x10001) Enter pass phrase for ca-key.pem: **** Verifying - Enter pass phrase for ca-key.pem: ****
We need to remember the passphrases set for our CA's private key, as we will always need them for the rest of this chapter.
- Next, we make sure that this private key is secure by restricting read and write access to us:
client$ chmod 600 ca-key.pem client$ ls -l ca-key.pem -rw-------. 1 dockeruser group 3326 Dec 2 20:45 ca-key.pem
- Finally, we will generate a certificate for our CA that is self-signed. Let's type the following command to place the self-signed certificate in a file called
ca.pem
:
client$ openssl req -key ca-key.pem -new -x509 \ -subj '/CN=Certificate Authority' \ -sha256 -days 365 -out ca.pem Enter pass phrase for ca-key.pem: ****
We now have a CA, and we will be distributing its ca.pem
certificate to our Docker host and client later in this section. For now, let's inspect the generate certificate:
client$ cat ca.pem -----BEGIN CERTIFICATE----- MIIFEzCCAvugAwIBAgIJAM19ce5sap+kMA0GCSqGSIb3DQEBCwUAMCAxHjAcBgNV BAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xODEyMDYwMzQ5MTNaFw0xOTEy MDYwMzQ5MTNaMCAxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTCCAiIw DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAKCESs7QpRZ78v8p2nKomCGABqCN b3E0vBpjveTnjA4kOEWVsHloq2o66yuuNff75GNWghzq791KyKJOy/dehNL9DauA DD3DJh0+uaOGn547W827Z37wJ64acNyvIQjyiyeLrpF4BzzxaZ/AJFVgqar5Kuqc qiOG3GUYcnfu6mpmlKoa1XqBtSQ+A2fd4/mpXC0zrDrz9MSEOCs5/Xm6/faexYae V8gBkCYWiUVUi+RRRc2vU1LzuiI5FsXmD3kNHCjNIbYIoyqKMzbTJjEffhN+5B/V Rc3qfRmfoEv8P0Hc4Wx55qH8BLWwhvNFAZ+nre+j7zPz+dTfLVyveOPxErHaI1V8 WH9qEVf+haNqUBrjNCuL+xyVNx7evPygD88jyZDWLK5Y0JTh2GSPqMeVi3hSKzNP GbVjT8tmkCUEsYbSJg2vkPYJR4aC8LLdJsjr7wkWBF1IcYYZpLo3EsUnkjNi7MGS pGdLob3UToekXaA1D6esDhlEB+3Tt/RWJkS91ijUiDs2kTSmDfnxUQGyeD4wx/rj lPFRSLdUUYiFcdI5VegZVSqYxW/Qw2/t+GvoLkrOrggqY1f++XUgK5hSoT8EqgiG SjapkgphMEquVP8UlZ3jC0VmgwFnRUEdqau6yLWMYG6TvLkyVi1Vmfam7CoB1aDn TccUszk+rezX+1nJAgMBAAGjUDBOMB0GA1UdDgQWBBTrqfPKO0i2peZ6Hd/BYOMq WXD9kDAfBgNVHSMEGDAWgBTrqfPKO0i2peZ6Hd/BYOMqWXD9kDAMBgNVHRMEBTAD AQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCa6SPGncEZSWu0WLfkh1mERa9JfBQzJFpv 1E7M3tZeFyJS7LfXdcf9WEAaWqTpha87A+5g9uBi/whYk47dyTik07/k+CyF112i 9GXK8j/UNCjAMOSluOCxpIsmMXp2Dn+ma21msN1K/lHK0ZhGWB9ZDggvdzRRPjic Dq3aQ49ATHQHGg9cqgZO0zXtcQYaHfCNds5YLNVL66eDhuN91V2MEqWtRDHfr0vA F3KldXfQ/clnrjGLqo7a3oR1R4QofQ03bV+PRIgub+l3Fee1D68BqF9dLRjUABd2 zm5OzNAmmHPSGWGvOxylvPrUS0ulUzMUWdoXN85SDdLHFXTXwpbD/GgqK+Y3BTgO 7d+mOoTHVEdw2gUXLaqeEchBge2Kh/LQtiN7Zp8OY7snX66Z8tF6W2MKhnSpDzcW J4WMbmaRqsTEeaRk0aTWkhBZukSZf4zjaa/abF+iRvU5c1OGS9GmYfuGq3Tlj+Xo JZNuKp9HzOPaj8qiD0DJW9EnuZ24zzpDSiSdmOdARcaaFFKhW8i+SVP6VqrAR3Nb OL8ne6w6kdoiq4+hPKfWVS9Yh0aQstJMNP91Nnw3J+aRz9eN03jpl/z18vHhW/xl nYJrB2KlC7SOnUT7TMJr4O5Aw1SidxMH6NLiiC1jbTWXDMuYL8UghDIk9Ne/WhBd qg0sW+boLw== -----END CERTIFICATE-----
Now that we have a CA for our PKI, we can use this CA to verify the identity of our Docker host. The following steps will prepare the identity of our Docker host:
- First, we log in to our Docker host. In here, we will generate a private key that will secure the remote API being served by running Docker Engine. The following command will save the private key into a file called
/etc/docker/server-key.pem
:
dockerhost$ openssl genrsa -out /etc/docker/server-key.pem 2048 Generating RSA private key, 2048 bit long modulus ................................+++ ........+++ e is 65537 (0x10001)
- Next, we make sure that this file is secure and can only be accessed by the Docker Engine daemon (through the root user):
dockerhost$ chmod 600 /etc/docker/server-key.pem dockerhost$ ls -l /etc/docker/server-key.pem -rw-------. 1 root root 1675 Dec 2 21:09 /etc/docker/server-key.pem
- Now that the private key is ready, we will use this file to generate a Certificate Signing Request(CSR).
openssl req
following command will generate a CSR:
dockerhost$ openssl req -key /etc/docker/server-key.pem -new -subj "/CN=dockerhost" -sha256 -out dockerhost.csr dockerhost$ ls -l dockerhost.csr -rw-r--r--. 1 root root 891 Dec 2 21:33 dockerhost.csr
- Next, we go back to our client workstation where our CA's files are hosted. In here, we will download the CSR from our Docker host:
client$ scp dockerhost:~/dockerhost.csr dockerhost.csr dockerhost.csr 100% 891 1.5MB/s 00:00
- We now prepare an OpenSSL configuration
server-ext.cnf
file that indicates that the certificates our CA will issue are used for server authentication:
extendedKeyUsage = serverAuth
- Finally, we can sign the CSR with our CA. The following command will place our Docker host's signed certificate in a file called
dockerhost.pem
:
client$ cd ~/ca client$ openssl x509 -req -CA ca.pem -CAkey ca-key.pem \ -CAcreateserial -extfile server-ext.cnf \ -in dockerhost.csr -out dockerhost.pem Signature ok subject=/CN=dockerhost Getting CA Private Key Enter pass phrase for ca-key.pem: ****
Now that we have the identity of our Docker host verified by our CA, we can now enable the secure TCP port in our Docker host. We will bring up the secure remote API with the following steps:
- Let's now go back into our Docker host. Here, we will copy the certificates of our Docker host and CA from our client workstation:
dockerhost$ scp client:~/ca/ca.pem /etc/docker/ca.pem ca.pem 100% 1911 1.1MB/s 00:00 dockerhost$ scp client:~/ca/dockerhost.pem /etc/docker/server.pem dockerhost.pem 100% 1428 1.2MB/s 00:00
- Now that our TLS assets are in place, let's now reconfigure the Docker Engine daemon file,
/etc/docker/daemon.json
, to use those certificates:
{ "tlsverify": true, "tlscacert": "/etc/docker/ca.pem", "tlskey": "/etc/docker/server-key.pem", "tlscert": "/etc/docker/server.pem" }
- Next, we configure the Docker Engine daemon to listen to a secure port by creating a
systemd
override file/etc/systemd/system/docker.service.d/override.conf
:
[Service] ExecStart= ExecStart=/usr/bin/dockerd -H unix:// -H tcp://0.0.0.0:2376
- Finally, we are now ready to restart Docker Engine:
dockerhost$ systemctl daemon-reload dockerhost$ systemctl restart docker.service
Our Docker host is now ready and serving a secure API.
Now that our Docker host is secure, it won't respond to requests from our Docker client yet. The Docker host will only respond to requests being made by clients verified by our CA.
The following steps will generate an identity for our Docker client:
- First, let's generate the private key of our Docker client in
~/.docker/key.pem
:
client$ openssl genrsa -out ~/.docker/key.pem 4096 Generating RSA private key, 4096 bit long modulus ...................++ .........................................................++ e is 65537 (0x10001)
- Next, we make sure that this private key is restricted to us for viewing:
client$ chmod 600 ~/.docker/key.pem
- We now generate the CSR for client in a file called
client.csr
:
client$ openssl req -subj '/CN=client' -new \ -key ~/.docker/key.pem -out client.csr
- Now that our CSR is ready, we will now create an OpenSSL configuration to indicate that certificates will be used for client authentication. The OpenSSL command following creates this configuration in a file called
~/ca/client-ext.cnf
:
extendedKeyUsage = clientAuth
- Finally, we are ready to issue the certificate for our Docker client. The following command writes our Docker client's certificate to
~/.docker/cert.pem
:
client$ openssl x509 -req -CA ca.pem -CAkey ca-key.pem -CAcreateserial -extfile client-ext.cnf -in ~/client.csr -out ~/.docker/cert.pem Signature ok subject=/CN=client Getting CA Private Key Enter pass phrase for ca-key.pem: ****
- To complete our client's TLS configuration, we will also deploy our CA's certificate in our
~/.docker directory
file:
client$ cp ca.pem ~/.docker/ca.pem
- Finally, we indicate to our Docker client that we will be connecting securely to our remote Docker host by exporting the following environment variables:
client$ export DOCKER_HOST=tcp://dockerhost:2376 client$ export DOCKER_TLS_VERIFY=true
Congratulations! We now have a secure communication channel between our Docker client and Docker host. To verify the connection, we can run the following command and show information about our remote Docker host:
client$ docker info
Docker introduced swarm
mode to its Docker engine from version 1.12.0. Docker Swarm allows us to pool together multiple Docker hosts to deploy our containers in a scalable and high availability way. In this section, we will build a small Docker Swarm cluster.
Let's dive into building our cluster with the following steps:
- First, we will go to our Docker host and initialize it as a
manager
.manager
is responsible for maintaining the state of our Docker Swarm cluster. It also dispatches tasks to other Docker hosts in our cluster. Let's type the following command to begin the initialization:
dockerhost$ docker swarm init Swarm initialized: current node (w49smc2ciy100gaecgx77yir3) is now a manager To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-4wbs...aq2r \ 172.16.132.187:2377
The preceding command generated a token that will be used by other Docker hosts to join our cluster.
- Next, we will go to a new Docker host called
node1
. We use the token from the previous step to make this Docker host join our Docker Swarm cluster as aworker
. Workers are members of the cluster that are responsible for running our containers. Let's now type the following command to make this new node join our cluster:
node1$ docker swarm join --token SWMTKN-1-4...aq2r \ 172.16.132.187:2377 This node joined a swarm as a worker.
Note
We can scale out our Docker Swarm cluster by adding more managers and workers using the same Docker Swarm join
command. More details can be found in the upstream Docker documentation at https://docs.docker.com/engine/swarm/join-nodes.
We have now finished setting up our Docker Swarm cluster. Let's go back to our Docker client workstation and confirm the members of our cluster:
client$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION w49smc * dockerhost Ready Active Leader 18.09.0 2e0aif node1 Ready Active 18.09.0
Hopefully, at this point, we have become familiar with interacting with Docker Engine. We have prepared a Docker host where we will run our containers. We built a PKI to ensure secure communication between our Docker host and our Docker client workstation, and we built a small Docker Swarm cluster consisting of multiple Docker hosts.
We accomplished all of this by logging in to servers and manually typing configuration commands. In the next chapter, we'll learn to automate provisioning these Docker hosts—and save us from typing!