Home Cloud & Networking Docker Deep Dive

Docker Deep Dive

By Nigel Poulton
books-svg-icon Book
eBook $27.99 $18.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $27.99 $18.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
About this book
A new version of this book is now available. Most applications, even the funky cloud-native microservices ones, need high-performance, production-grade infrastructure to run on. Having impeccable knowledge of Docker will help you thrive in the modern cloud-first world. With this book, you will gain the skills you need in order to work with Docker and its containers. The book begins with an introduction to containers and explains their functionality and application in the real world. You will then get an overview of VMware, Kubernetes, and Docker and learn to install Docker on Windows, Mac, and Linux. Once you have understood the Ops and Dev perspective of Docker, you will be able to see the big picture and understand what Docker exactly does. The book then turns its attention to the more technical aspects, guiding you through practical exercises covering Docker engine, Docker images, and Docker containers. You will learn techniques for containerizing an app, deploying apps with Docker Compose, and managing cloud-native applications with Swarm. You will also build Docker networks and Docker overlay networks and handle applications that write persistent data. Finally, you will deploy apps with Docker stacks and secure your Docker environment. By the end of this book, you will be well-versed in Docker and containers and have developed the skills to create, deploy, and run applications on the cloud.
Publication date:
October 2020
Publisher
Packt
Pages
250
ISBN
9781800565135

 

Part 1: The big picture stuff

 

1: Containers from 30,000 feet

Containers are definitely a thing.

In this chapter we’ll get into things like; why we have containers, what they do for us, and where we can use them.

The bad old days

Applications are at the heart of businesses. If applications break, businesses break. Sometimes they even go bust. These statements get truer every day!

Most applications run on servers. In the past we could only run one application per server. The open-systems world of Windows and Linux just didn’t have the technologies to safely and securely run multiple applications on the same server.

As a result, the story went something like this… Every time the business needed a new application, the IT department would buy a new server. Most of the time nobody knew the performance requirements of the new application, forcing the IT department to make guesses when choosing the model and size of the server to buy.

As a result, IT did the only thing it could do — it bought big fast servers that cost a lot of money. After all, the last thing anyone wanted, including the business, was under-powered servers unable to execute transactions and potentially losing customers and revenue. So, IT bought big. This resulted in over-powered servers operating as low as 5-10% of their potential capacity. A tragic waste of company capital and environmental resources!

Hello VMware!

Amid all of this, VMware, Inc. gave the world a gift — the virtual machine (VM). And almost overnight, the world changed into a much better place. We finally had a technology that allowed us to safely and securely run multiple business applications on a single server. Cue wild celebrations!

This was a game changer. IT departments no longer needed to procure a brand-new oversized server every time the business needed a new application. More often than not, they could run new apps on existing servers that were sitting around with spare capacity.

All of a sudden, we could squeeze massive amounts of value out of existing corporate assets, resulting in a lot more bang for the company’s buck ($).

VMwarts

But… and there’s always a but! As great as VMs are, they’re far from perfect!

The fact that every VM requires its own dedicated operating system (OS) is a major flaw. Every OS consumes CPU, RAM and other resources that could otherwise be used to power more applications. Every OS needs patching and monitoring. And in some cases, every OS requires a license. All of this results in wasted time and resources.

The VM model has other challenges too. VMs are slow to boot, and portability isn’t great — migrating and moving VM workloads between hypervisors and cloud platforms is harder than it needs to be.

Hello Containers!

For a long time, the big web-scale players, like Google, have been using container technologies to address the shortcomings of the VM model.

In the container model, the container is roughly analogous to the VM. A major difference is that containers do not require their own full-blown OS. In fact, all containers on a single host share the host’s OS. This frees up huge amounts of system resources such as CPU, RAM, and storage. It also reduces potential licensing costs and reduces the overhead of OS patching and other maintenance. Net result: savings on the time, resource, and capital fronts.

Containers are also fast to start and ultra-portable. Moving container workloads from your laptop, to the cloud, and then to VMs or bare metal in your data center is a breeze.

Linux containers

Modern containers started in the Linux world and are the product of an immense amount of work from a wide variety of people over a long period of time. Just as one example, Google LLC has contributed many container-related technologies to the Linux kernel. Without these, and other contributions, we wouldn’t have modern containers today.

Some of the major technologies that enabled the massive growth of containers in recent years include; kernel namespaces, control groups, union filesystems, and of course Docker. To re-emphasize what was said earlier — the modern container ecosystem is deeply indebted to the many individuals and organizations that laid the strong foundations that we currently build on. Thank you!

Despite all of this, containers remained complex and outside of the reach of most organizations. It wasn’t until Docker came along that containers were effectively democratized and accessible to the masses.

Note: There are many operating system virtualization technologies similar to containers that pre-date Docker and modern containers. Some even date back to System/360 on the Mainframe. BSD Jails and Solaris Zones are some other well-known examples of Unix-type container technologies. However, in this book we are restricting our conversation to modern containers made popular by Docker.

Hello Docker!

We’ll talk about Docker in a bit more detail in the next chapter. But for now, it’s enough to say that Docker was the magic that made Linux containers usable for mere mortals. Put another way, Docker, Inc. made containers simple!

Windows containers

Over the past few years, Microsoft Corp. has worked extremely hard to bring Docker and container technologies to the Windows platform.

At the time of writing, Windows containers are available on the Windows desktop and Windows Server platforms (certain versions of Windows 10 and later, and Windows Server 2016 and later). In achieving this, Microsoft has worked closely with Docker, Inc. and the open-source community.

The core Windows kernel technologies required to implement containers are collectively referred to as Windows Containers. The user-space tooling to work with these Windows Containers can be Docker. This makes the Docker experience on Windows almost exactly the same as Docker on Linux. This way developers and sysadmins familiar with the Docker toolset from the Linux platform can feel at home using Windows containers.

This revision of the book includes a mix of Linux and Windows examples.

Windows containers vs Linux containers

It’s vital to understand that a running container shares the kernel of the host machine it is running on. This means that a containerized Windows app will not run on a Linux-based Docker host, and vice-versa — Windows containers require a Windows host, and Linux containers require a Linux host. Only… it’s not always that simple.

It is possible to run Linux containers on Windows machines. For example, Docker Desktop running on Windows has two modes — “Windows containers” and “Linux containers”. Depending on your version of Docker Desktop, Linux container run either inside a lightweight Hyper-V VM or using the Windows Subsystem for Linux (WSL). The WSL option is newer and the strategic option for the future as it doesn’t require a Hyper-V VM and offers better performance and compatibility.

What about Mac containers?

There is currently no such thing as Mac containers.

However, you can run Linux containers on your Mac using Docker Desktop. This works by seamlessly running your containers inside of a lightweight Linux VM on your Mac. It’s extremely popular with developers, who can easily develop and test Linux containers on their Mac.

What about Kubernetes

Kubernetes is an open-source project out of Google that has quickly emerged as the de facto orchestrator of containerized apps. That’s just a fancy way of saying Kubernetes is the most popular tool for deploying and managing containerized apps.

Note: A containerized app is an application running as a container.

At the time of writing, Kubernetes uses Docker as its default container runtime — the low-level technology that pulls images and starts and stops containers. However, Kubernetes has a pluggable container runtime interface (CRI) that makes it easy to swap-out Docker for a different container runtime. In the future, Docker might be replaced by containerd as the default container runtime in Kubernetes. More on containerd later in the book, but for now it’s enough to know that containerd is the small specialized part of Docker that does the low-level tasks of starting and stopping containers.

The important thing to know about Kubernetes, at this stage, is that it’s a higher-level platform than Docker, and it currently uses Docker for its low-level container-related operations.

I have the following resources to help you learn Kubernetes:

  • The Kubernetes Book
  • Getting Started with Kubernetes video course
  • Kubernetes 101 video course

Getting Started with Kubernetes is available at pluralsight.com and Kubernetes 101 is available at udemy.com.

Chapter Summary

We used to live in a world where every time the business wanted a new application we had to buy a brand-new server. VMware came along and enabled us to drive more value out of new and existing company IT assets. As good as VMware and the VM model is, it’s not perfect. Following the success of VMware and hypervisors came a newer more efficient and lightweight virtualization technology called containers. But containers were initially hard to implement and were only found in the data centers of web giants that had Linux kernel engineers on staff. Along came Docker, Inc. and suddenly containers were available to the masses.

Speaking of Docker… let’s go find who, why, and what Docker is!

 

2: Docker

No book or conversation about containers is complete without talking about Docker. But when we say “Docker”, we can be referring to either of the following:

  1. Docker, Inc. the company
  2. Docker the technology

Docker - The TLDR

Docker is software that runs on Linux and Windows. It creates, manages, and can even orchestrate containers. The software is currently built from various tools from the Moby open-source project. Docker, Inc. is the company that created the technology and continues to create technologies and solutions that make it easier to get the code on your laptop running in the cloud.

That’s the quick version. Let’s dive a bit deeper.

Docker, Inc.

Docker, Inc. is a San Francisco based technology company founded by French-born American developer and entrepreneur Solomon Hykes. Solomon is no longer at the company.

Figure 2.1 Docker, Inc. logo.
Figure 2.1 Docker, Inc. logo.

The company started out as a platform as a service (PaaS) provider called dotCloud. Behind the scenes, the dotCloud platform was built on Linux containers. To help create and manage these containers, they built an in-house tool that they eventually nick-named “Docker”. And that’s how the Docker technology was born!

It’s also interesting to know that the word “Docker” comes from a British expression meaning dock worker — somebody who loads and unloads cargo from ships.

In 2013 they got rid of the struggling PaaS side of the business, rebranded the company as “Docker, Inc.”, and focussed on bringing Docker and containers to the world. They were immensely successfully in bringing containers into mainstream IT, but so far they’ve struggled to make a profitable business.

At the time of writing, Docker, Inc. is focussing on their Docker Desktop and Docker Hub products to streamline the process of getting from source code on a laptop, all the way to a running application in the cloud.

Throughout this book we’ll use the term “Docker, Inc.” when referring to Docker the company. All other uses of the term “Docker” will refer to the technology.

The Docker technology

When most people talk about Docker, they’re referring to the technology that runs containers. However, there are at least three things to be aware of when referring to Docker as a technology:

  1. The runtime
  2. The daemon (a.k.a. engine)
  3. The orchestrator

Figure 2.2 shows the three layers and will be a useful reference as we explain each component. We’ll get deeper into each later in the book.

Figure 2.2 Docker architecture.
Figure 2.2 Docker architecture.

The runtime operates at the lowest level and is responsible for starting and stopping containers (this includes building all of the OS constructs such as namespaces and cgroups). Docker implements a tiered runtime architecture with high-level and low-level runtimes that work together.

The low-level runtime is called runc and is the reference implementation of Open Containers Initiative (OCI) runtime-spec. Its job is to interface with the underlying OS and start and stop containers. Every running container on a Docker node has a runc instance managing it.

The higher-level runtime is called containerd. containerd does a lot more than runc. It manages the entire lifecycle of a container, including pulling images, creating network interfaces, and managing lower-level runc instances. containerd is pronounced “container-dee’ and is a graduated CNCF project used by Docker and Kubernetes as a container runtime.

A typical Docker installation has a single containerd process (docker-containerd) controlling the runc (docker-runc) instances associated with each running container.

The Docker daemon (dockerd) sits above containerd and performs higher-level tasks such as; exposing the Docker remote API, managing images, managing volumes, managing networks, and more…

A major job of the Docker daemon is to provide an easy-to-use standard interface that abstracts the lower levels.

Docker also has native support for managing clusters of nodes running Docker. These clusters are called swarms and the native technology is called Docker Swarm. Docker Swarm is easy-to-use and many companies are using it in real-world production. However, most people are choosing to use Kubernetes instead of Docker Swarm.

The Open Container Initiative (OCI)

Earlier in the chapter we mentioned the Open Containers Initiative — OCI.

The OCI is a governance council responsible for standardizing the low-level fundamental components of container infrastructure. In particular it focusses on image format and container runtime (don’t worry if you’re not comfortable with these terms yet, we’ll cover them in the book).

It’s also true that no discussion of the OCI is complete without mentioning a bit of history. And as with all accounts of history, the version you get depends on who’s doing the talking. So, this is container history according to Nigel :-D

From day one, use of Docker grew like crazy. More and more people used it in more and more ways for more and more things. So, it was inevitable that some parties would get frustrated. This is normal and healthy.

The TLDR of this history according to Nigel is that a company called CoreOS (acquired by Red Hat which was then acquired by IBM) didn’t like the way Docker did certain things. So, they created an open standard called appc that defined things like image format and container runtime. They also created an implementation of the spec called rkt (pronounced “rocket”).

This put the container ecosystem in an awkward position with two competing standards.

Getting back to the story, this threatened to fracture the ecosystem and present users and customers with a dilemma. While competition is usually a good thing, competing standards is usually not. They cause confusion and slowdown user adoption. Not good for anybody.

With this in mind, everybody did their best to act like adults and came together to form the OCI — a lightweight agile council to govern container standards.

At the time of writing, the OCI has published two specifications (standards) -

An analogy that’s often used when referring to these two standards is rail tracks. These two standards are like agreeing on standard sizes and properties of rail tracks, leaving everyone else free to build better trains, better carriages, better signalling systems, better stations… all safe in the knowledge that they’ll work on the standardized tracks. Nobody wants two competing standards for rail track sizes!

It’s fair to say that the two OCI specifications have had a major impact on the architecture and design of the core Docker product. As of Docker 1.11, the Docker Engine architecture conforms to the OCI runtime spec.

The OCI is organized under the auspices of the Linux Foundation.

Chapter summary

In this chapter, we learned about Docker, Inc. the company, and the Docker technology.

Docker, Inc. is a technology company out of San Francisco with an ambition to change the way we do software. They were arguably the first-movers and instigators of the modern container revolution.

The Docker technology focuses on running and managing application containers. It runs on Linux and Windows, can be installed almost anywhere, and is currently the most popular container runtime used by Kubernetes.

The Open Container Initiative (OCI) was instrumental in standardizing the container runtime format and container image format.

 

3: Installing Docker

There are lots of ways and places to install Docker. There’s Windows, Mac, and Linux. You can install in the cloud, on premises, and on your laptop. And there are manual installs, scripted installs, wizard-based installs… There literally are loads of ways and places to install Docker.

But don’t let that scare you. They’re all really easy, and a simple search for “how to install docker on <insert your choice here>” will reveal up-to-date instructions that are easy to follow. As a result, we won’t waste too much space here. We’ll cover the following.

  • Docker Desktop installs on
    • Windows 10
    • Mac
  • Server installs on
    • Linux
    • Windows Server 2019
  • Play with Docker

Docker Desktop

Docker Desktop is a packaged product from Docker, Inc. It runs on 64-bit versions of Windows 10 and Mac, and it’s easy to download and install.

Once the installation is complete, you have a single-engine Docker environment that is great for development purposes. It includes Docker Compose and you can choose to enable a single-node Kubernetes cluster.

Early versions of Docker Desktop experienced some feature-lag while the product was developed with a stability first, features second approach. However, the product is now mature and a key technology in Docker, Inc’s focus on making it easier to get from the source code on your laptop to running applications in the cloud.

Docker Desktop on Windows 10 can run native Windows containers as well as Linux containers. Docker Desktop on Mac can only run Linux containers.

Windows pre-reqs

Docker Desktop on Windows requires all of the following:

  • 64-bit version of Windows 10 Pro/Enterprise/Education (does not work with Home edition)
  • Hardware virtualization support must be enabled in your system’s BIOS
  • The Hyper-V and Containers features must be enabled in Windows

The installer can enable the Hyper-V and Containers features, but it’s your responsibility to enable hardware virtualization in your BIOS (be very careful changing anything in your system’s BIOS).

Installing Docker Desktop on Windows 10

Perform a google search for “install Docker Desktop”. This will take you to the relevant download page where you can download the installer and follow the instructions. It’s that simple!

At the time of writing, you can choose between the stable channel and the edge channel. The names are self-explanatory, with the edge channel providing earlier access bleeding-edge features.

Once the installation is complete you may have to manually start Docker Desktop from the Windows Start menu. It can take a minute for it to start, and you can watch the start progress via the animated whale icon on the Windows task bar at the bottom of the screen.

Once it’s up and running you can open a PowerShell prompt and type some simple docker commands.

$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:23:10 2020
 OS/Arch:           windows/amd64
 Experimental:      true
Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:29:16 2020
  OS/Arch:          linux/amd64
  Experimental:     true
<Snip>

Notice the output is showing OS/Arch: linux/amd64 for the Server component. This is because a default installation assumes you’ll be working with Linux containers. It does this by running the Docker daemon inside of a lightweight Linux Hyper-V VM.

Switching to Windows containers is as simple as right-clicking the Docker whale icon in the Windows notifications tray and selecting Switch to Windows containers.... You can achieve the same thing from the command line with the following command (located in the \Program Files\Docker\Docker directory):

C:\Program Files\Docker\Docker> .\dockercli -SwitchDaemon

Be aware that any existing Linux containers will keep running in the background, but you won’t be able to see or manage them until you switch back to Linux containers mode.

You’ll be prompted to enable the Windows Containers feature if it isn’t already enabled.

Run another docker version command and look for the windows/amd64 line in the Server section of the output.

C:\> docker version
Client:
 <Snip>

Server:
 Engine:
  Version:      19.03.8
  API version:  1.40 (minimum version 1.24)
  Go version:   go1.12.17
  Git commit:   afacb8b
  Built:        Wed Mar 11 01:37:20 2020
  OS/Arch:      windows/amd64
  Experimental: true

You can now run and manage Windows containers (containers running Windows applications).

Congratulations. You now have a working installation of Docker on your Windows 10 machine.

Installing Docker Desktop on Mac

Docker Desktop for Mac is like Docker Desktop on Windows 10 — a packaged product from Docker, Inc with a simple installer that gets you a single-engine installation of Docker that’s ideal for local development needs. You can also enable a single-node Kubernetes cluster.

We’ll look at a simple installation in a second, but before doing that it’s worth noting that Docker Desktop on Mac doesn’t give you the Docker Engine running natively on the Mac OS Darwin kernel. Behind the scenes, the Docker daemon is running inside a lightweight Linux VM that seamlessly exposes the daemon and API to your Mac environment. This means you can open a terminal on your Mac and use the regular Docker commands.

Although this works seamlessly on your Mac, don’t forget that it’s Docker on Linux under the hood — so it’s only going work with Linux-based Docker containers. This is good though, as it’s where most of the container action is.

Figure 3.1 shows the high-level architecture for Docker Desktop on Mac.

Figure 3.1
Figure 3.1

The simplest way to install Docker Desktop on your Mac is perform a google search for “install Docker Desktop”. Follow the links to the download page where you can download the installer and follow the instructions. It’s that simple.

As with Docker Desktop on Windows 10, you can choose the stable channel or the edge channel — the edge channel providing earlier access bleeding-edge features.

Download the installer and follow the step-by-step instructions.

Once the installation is complete you may have to manually start Docker Desktop from the MacOS Launchpad. It can take a minute for it to start, and you can watch the animated Docker whale icon in the status bar at the top of your screen. Once Docker Desktop is started, the whale will stop being animated. You can click the whale icon to manage Docker Desktop.

Open a terminal window and run some regular Docker commands. Try the following.

$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:21:11 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:29:16 2020
  OS/Arch:          linux/amd64
  Experimental:     true

Notice that the OS/Arch: for the Server component is showing as linux/amd64. This is because the daemon is running inside of the Linux VM we mentioned earlier. The Client component is a native Mac application and runs directly on the Mac OS Darwin kernel (OS/Arch: darwin/amd64).

You can now use Docker on your Mac.

Installing Docker on Linux

There are lots of ways to install Docker on Linux and most of them are easy. The hardest part is usually deciding which Linux distro to use.

The internet has lots of guides for installing and working with Docker on many distributions of Linux. In this section we’ll look at one of the ways to install on Ubuntu Linux 20.04 LTS. The procedure assumes you’ve already installed Linux and are logged on.

  1. Update the apt package index.
     $ sudo apt-get update
     Get:1 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu focal InRelease [265 kB]
     ...
    
  2. Install Docker from the official repo.
     $ sudo apt-get install docker.io
     Reading package lists... Done
     Building dependency tree   
     ...
    

Docker is now installed and you can test by running some commands.

$ sudo docker --version
Docker version 19.03.8, build afacb8b7f0

$ sudo docker info
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
  ...

Installing Docker on Windows Server 2019

Most of the public cloud platforms offer off-the-shelf copies of Windows Server 2019 with Docker pre-installed. Simply choose one of these – such as Microsoft Windows Server 2019 Base with Containers - ami-0b809eef92577a4f1 on AWS – and you’re good to go.

Installing Docker on other versions of Windows Server 2019 is incredibly easy.

The following procedure assumes you’ve installed Windows Server 2019 and are logged on with administrator privileges.

  1. Install the Docker Provider

    Run this command from a PowerShell terminal.

     PS C:\> Install-Module DockerMsftProvider -Force
     NuGet provider is required to continue
     PowerShellGet requires NuGet provider version
     <Snip>
     Do you want PowerShellGet to install and import the NuGet provider now?
     [Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): y
    
  2. Install Docker
     PS C:\> Install-Package Docker -ProviderName DockerMsftProvider -Force
     WARNING: A restart is required to enable the containers feature. Please restart.
     Name       Version     Source           Summary
     ----       -------     ------           -------
     Docker     19.03.5     DockerDefault    Contains Docker EE for use with Windows Server.
    
  3. Restart your machine

Congratulations, Docker is now installed and configured to automatically start when the system boots.

Run some commands to verify Docker is working.

PS C:\> docker version
Client: Docker Engine - Enterprise
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        2ee0c57608
 Built:             11/13/2019 08:00:16
 OS/Arch:           windows/amd64
 Experimental:      false
Server: Docker Engine - Enterprise
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.12.12
  Git commit:       2ee0c57608
  Built:            11/13/2019 07:58:51
  OS/Arch:          windows/amd64
  Experimental:     false

Docker is now installed and you are ready to start using Windows containers.

Play with Docker

Play with Docker (PWD) provides a free-to-use fully functional Docker playground that lasts for 4 hours. You can add multiple nodes and even cluster them in a swarm.

Sometimes performance can be slow, but for a free-to-use service it is excellent!

Visit https://labs.play-with-docker.com/

Chapter Summary

You can run Docker almost anywhere and most of the installation methods are simple.

Docker Desktop provides you a single-engine Docker environment on your Mac or Windows 10 laptop. It’s simple to install, is intended for development activities and even allows you to spin-up a single-node Kubernetes cluster.

Docker can be installed on Windows Server and Linux, with most operating systems having packages that are simple to install.

Play with Docker is a free 4-hour Docker playground on the internet.

 

4: The big picture

The aim of this chapter is to paint a quick big-picture of what Docker is all about before we dive in deeper in later chapters.

We’ll break this chapter into two:

  • The Ops perspective
  • The Dev perspective

In the Ops Perspective section, we’ll download an image, start a new container, log in to the new container, run a command inside of it, and then destroy it.

In the Dev Perspective section, we’ll focus more on the app. We’ll clone some app-code from GitHub, inspect a Dockerfile, containerize the app, run it as a container.

These two sections will give you a good idea of what Docker is all about and how the major components fit together. It’s recommended that you read both sections to get the dev and the ops perspectives. DevOps anyone?

Don’t worry if some of the stuff we do here is totally new to you. We’re not trying to make you an expert in this chapter. This is about giving you a feel of things — setting you up so that when we get into the details in later chapters, you have an idea of how the pieces fit together.

If you want to follow along, all you need is a single Docker host with an internet connection. I recommend Docker Desktop for your Mac or Windows PC. However, the examples will work anywhere that you have Docker installed. We’ll be showing examples using Linux containers and Windows containers.

If you can’t install software and don’t have access to a public cloud, another great way to get Docker is Play With Docker (PWD). This is a web-based Docker playground that you can use for free. Just point your web browser to https://labs.play-with-docker.com/ and you’re ready to go (you’ll need a Docker Hub or GitHub account to be able to login).

As we progress through the chapter, we may use the terms “Docker host” and “Docker node” interchangeably. Both refer to the system that you are running Docker on.

The Ops Perspective

When you install Docker, you get two major components:

  • the Docker client
  • the Docker daemon (sometimes called the “Docker engine”)

The daemon implements the runtime, API and everything else required to run Docker.

In a default Linux installation, the client talks to the daemon via a local IPC/Unix socket at /var/run/docker.sock. On Windows this happens via a named pipe at npipe:////./pipe/docker_engine. Once installed, you can use the docker version command to test that the client and daemon (server) are running and talking to each other.

> docker version
Client: Docker Engine - Community
 Version:       19.03.8
 API version:   1.40
 Go version:    go1.12.17
 Git commit:    afacb8b
 Built: Wed Mar 11 01:23:10 2020
 OS/Arch:       linux/amd64
 Experimental:  false

Server:
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:29:16 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  ...

If you get a response back from the Client and Server, you’re good to go.

If you are using Linux and get an error response from the Server component, make sure that Docker is up and running. Also, try the command again with sudo in front of it: sudo docker version. If it works with sudo you will need to add your user account to the local docker group, or prefix the remainder of the commands in the book with sudo.

Images

It’s useful to think of a Docker image as an object that contains an OS filesystem, an application, and all application dependencies. If you work in operations, it’s like a virtual machine template. A virtual machine template is essentially a stopped virtual machine. In the Docker world, an image is effectively a stopped container. If you’re a developer, you can think of an image as a class.

Run the docker image ls command on your Docker host.

$ docker image ls
REPOSITORY    TAG        IMAGE ID       CREATED       SIZE

If you are working from a freshly installed Docker host, or Play With Docker, you will have no images and it will look like the previous output.

Getting images onto your Docker host is called “pulling”. If you are following along with Linux, pull the ubuntu:latest image. If you are following along on Windows, pull the mcr.microsoft.com/powershell:lts-nanoserver-1903 image.

$ docker image pull ubuntu:latest
latest: Pulling from library/ubuntu
50aff78429b1: Pull complete
f6d82e297bce: Pull complete
275abb2c8a6f: Pull complete
9f15a39356d6: Pull complete
fc0342a94c89: Pull complete
Digest: sha256:fbaf303...c0ea5d1212
Status: Downloaded newer image for ubuntu:latest

Windows images can be large and take a long time to pull.

Run the docker image ls command again to see the image you just pulled.

$ docker images
REPOSITORY      TAG      IMAGE ID        CREATED         SIZE
ubuntu          latest   1d622ef86b13    16 hours ago    73.9MB

We’ll get into the details of where the image is stored and what’s inside of it in later chapters. For now, it’s enough to know that an image contains enough of an operating system (OS), as well as all the code and dependencies to run whatever application it’s designed for. The ubuntu image that we’ve pulled has a stripped-down version of the Ubuntu Linux filesystem, including a few of the common Ubuntu utilities. The mcr.microsoft.com/powershell:lts-nanoserver-1903 image contains a Windows Server Core OS plus PowerShell.

If you pull an application container such as nginx or mcr.microsoft.com/windows/servercore/iis, you will get an image that contains some OS, as well as the code to run either NGINX or IIS.

It’s also worth noting that each image gets its own unique ID. When referencing images, you can refer to them using either IDs or names. If you’re working with image ID’s, it’s usually enough to type the first few characters of the ID — as long as it’s unique, Docker will know which image you mean.

Containers

Now that we have an image pulled locally, we can use the docker container run command to launch a container from it.

For Linux:

$ docker container run -it ubuntu:latest /bin/bash
root@6dc20d508db0:/#

For Windows:

> docker container run -it mcr.microsoft.com/powershell:lts-nanoserver-1903 pwsh.exe

PowerShell 7.0.0
Copyright (C) Microsoft Corporation. All rights reserved.
PS C:\>

Look closely at the output from the previous commands. You should notice that the shell prompt has changed in each instance. This is because the -it flags switch your shell into the terminal of the container — you are literally inside of the new container!

Let’s examine that docker container run command.

docker container run tells the Docker daemon to start a new container. The -it flags tell Docker to make the container interactive and to attach the current shell to the container’s terminal (we’ll get more specific about this in the chapter on containers). Next, the command tells Docker that we want the container to be based on the ubuntu:latest image (or the mcr.microsoft.com/powershell:lts-nanoserver-1903 image if you’re following along with Windows). Finally, we tell Docker which process we want to run inside of the container. For the Linux example we’re running a Bash shell, for the Windows container we’re running PowerShell.

Run a ps command from inside of the container to list all running processes.

Linux example:

root@6dc20d508db0:/# ps -elf
F S UID    PID  PPID   NI ADDR SZ WCHAN  STIME TTY      TIME CMD
4 S root     1     0    0 -  4560 -      13:38 pts/0    00:00:00 /bin/bash
0 R root     9     1    0 -  8606 -      13:38 pts/0    00:00:00 ps -elf

Windows example:

PS C:\> ps

NPM(K)    PM(M)      WS(M)     CPU(s)      Id  SI ProcessName
 ------    -----      -----     ------      --  -- -----------
      5     0.90       3.78       0.00    1068   1 CExecSvc
      6     0.97       4.12       0.03    1184   1 conhost
      6     0.87       2.16       0.00     972   1 csrss
      0     0.06       0.01       0.00       0   0 Idle
     18     4.38      12.32       0.00     272   1 lsass
     54    34.82      65.09       1.27    1212   1 pwsh
      9     1.61       4.99       0.00    1020   1 services
      4     0.49       1.18       0.00     948   0 smss
     14     1.98       6.61       0.00     628   1 svchost
     12     2.95      10.02       0.00     752   1 svchost
      8     1.83       6.02       0.00     788   1 svchost
      7     1.42       4.70       0.00    1040   1 svchost
     16     6.12      11.41       0.00    1120   1 svchost
     24     3.73      10.38       0.00    1168   1 svchost
     15     9.60      18.96       0.00    1376   1 svchost
      0     0.16       0.14       0.00       4   0 System
      8     1.16       4.12       0.00    1004   1 wininit

The Linux container only has two processes:

  • PID 1. This is the /bin/bash process that we told the container to run with the docker container run command.
  • PID 9. This is the ps -elf command/process that we ran to list the running processes.

The presence of the ps -elf process in the Linux output can be a bit misleading as it is a short-lived process that dies as soon as the ps command completes. This means the only long-running process inside of the container is the /bin/bash process.

The Windows container has a lot more going on. This is an artefact of the way the Windows Operating System works. However, even though the Windows container has a lot more processes than the Linux container, it is still a lot less than a regular Windows Server.

Press Ctrl-PQ to exit the container without terminating it. This will land your shell back at the terminal of your Docker host. You can verify this by looking at your shell prompt.

Now that you are back at the shell prompt of your Docker host, run the ps command again.

Notice how many more processes are running on your Docker host compared to the container you just ran. Windows containers run far fewer processes than Windows hosts, and Linux containers run far less than Linux hosts.

In a previous step, you pressed Ctrl-PQ to exit from the container. Doing this from inside of a container will exit you from the container without killing it. You can see all running containers on your system using the docker container ls command.

$ docker container ls
CONTAINER ID   IMAGE          COMMAND       CREATED  STATUS    NAMES
6dc20d508db0   ubuntu:latest  "/bin/bash"   7 mins   Up 7 min  vigilant_borg

The output above shows a single running container. This is the container that you created earlier. The presence of the container in this output proves that it’s still running. You can also see that it was created 7 minutes ago and has been running for 7 minutes.

Attaching to running containers

You can attach your shell to the terminal of a running container with the docker container exec command. As the container from the previous steps is still running, let’s make a new connection to it.

Linux example:

This example references a container called “vigilant_borg”. The name of your container will be different, so remember to substitute “vigilant_borg” with the name or ID of the container running on your Docker host.

$ docker container exec -it vigilant_borg bash
root@6dc20d508db0:/#

Windows example:

This example references a container called “pensive_hamilton”. The name of your container will be different, so remember to substitute “pensive_hamilton” with the name or ID of the container running on your Docker host.

> docker container exec -it pensive_hamilton pwsh.exe

PowerShell 7.0.0
Copyright (C) Microsoft Corporation. All rights reserved.
PS C:\>

Notice that your shell prompt has changed again. You are logged into the container again.

The format of the docker container exec command is: docker container exec <options> <container-name or container-id> <command/app>. In our examples, we used the -it options to attach our shell to the container’s shell. We referenced the container by name, and told it to run the bash shell (PowerShell in the Windows example). We could easily have referenced the container by its hex ID.

Exit the container again by pressing Ctrl-PQ.

Your shell prompt should be back to your Docker host.

Run the docker container ls command again to verify that your container is still running.

$ docker container ls
CONTAINER ID   IMAGE          COMMAND       CREATED  STATUS    NAMES
6dc20d508db0   ubuntu:latest  "/bin/bash"   9 mins   Up 9 min  vigilant_borg

Stop the container and kill it using the docker container stop and docker container rm commands. Remember to substitute the names/IDs of your own containers.

$ docker container stop vigilant_borg
vigilant_borg

$ docker container rm vigilant_borg
vigilant_borg

Verify that the container was successfully deleted by running the docker container ls command with the -a flag. Adding -a tells Docker to list all containers, even those in the stopped state.

$ docker container ls -a
CONTAINER ID    IMAGE    COMMAND    CREATED    STATUS    PORTS    NAMES

You’ve just pulled a Docker image, started a container from it, attached to it, executed a command inside it, stopped it, and deleted it.

The Dev Perspective

Containers are all about the apps.

In this section, we’ll clone an app from a Git repo, inspect its Dockerfile, containerize it, and run it as a container.

The Linux app can be cloned from: https://github.com/nigelpoulton/psweb.git

The Windows app can be cloned from: https://github.com/nigelpoulton/win-web.git

The rest of this section will focus on the Linux NGINX example. However, both examples are containerizing simple web apps, so the process is the same. Where there are differences in the Windows example we will highlight them to help you follow along.

Run all of the following commands from a terminal on your Docker host.

Clone the repo locally. This will pull the application code to your local Docker host ready for you to containerize it.

Be sure to substitute the following repo with the Windows repo if you are following along with the Windows example.

$ git clone https://github.com/nigelpoulton/psweb.git
Cloning into 'psweb'...
remote: Counting objects: 15, done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 15 (delta 2), reused 15 (delta 2), pack-reused 0
Unpacking objects: 100% (15/15), done.
Checking connectivity... done.

Change directory into the cloned repo’s directory and list its contents.

$ cd psweb
$ ls -l
total 40
-rw-r--r--@ 1 ubuntu ubuntu  338 24 Apr 19:29 Dockerfile
-rw-r--r--@ 1 ubuntu ubuntu  396 24 Apr 19:32 README.md
-rw-r--r--@ 1 ubuntu ubuntu  341 24 Apr 19:29 app.js
-rw-r--r--  1 ubuntu ubuntu  216 24 Apr 19:29 circle.yml
-rw-r--r--@ 1 ubuntu ubuntu  377 24 Apr 19:36 package.json
drwxr-xr-x  4 ubuntu ubuntu  128 24 Apr 19:29 test
drwxr-xr-x  3 ubuntu ubuntu   96 24 Apr 19:29 views

The Linux example is a simple nodejs web app. The Windows example is an IIS server running some static HTML.

Both Git repos contain a file called Dockerfile. This is a plain-text document that tells Docker how to build an app and dependencies into a Docker image.

List the contents of the Dockerfile.

$ cat Dockerfile

FROM alpine
LABEL maintainer="nigelpoulton@hotmail.com"
RUN apk add --update nodejs nodejs-npm
COPY . /src
WORKDIR /src
RUN  npm install
EXPOSE 8080
ENTRYPOINT ["node", "./app.js"]

The contents of the Dockerfile in the Windows example are different. However, this isn’t important at this stage. For now, it’s enough to understand that each line represents an instruction that Docker uses to build an image.

At this point we have pulled some application code from a remote Git repo. We also have a Dockerfile containing instructions on how to build the app into a Docker image.

Use the docker image build command to create a new image using the instructions in the Dockerfile. This example creates a new Docker image called test:latest.

The command is the same for the Linux and Windows examples, and be sure to run it from within the directory containing the app code and Dockerfile.

$ docker image build -t test:latest .

Sending build context to Docker daemon  74.75kB
Step 1/8 : FROM alpine
latest: Pulling from library/alpine
88286f41530e: Pull complete
Digest: sha256:f006ecbb824...0c103f4820a417d
Status: Downloaded newer image for alpine:latest
 ---> 76da55c8019d
<Snip>
Successfully built f154cb3ddbd4
Successfully tagged test:latest

Note: It may take a long time for the build to finish in the Windows example. This is because of the image being pulled is several gigabytes in size.

Once the build is complete, check to make sure that the new test:latest image exists on your host.

$ docker image ls
REPO     TAG      IMAGE ID        CREATED         SIZE
test     latest   f154cb3ddbd4    1 minute ago    81.5MB
...

You have a newly-built Docker image with the app and dependencies inside.

Run a container from the image and test the app.

Linux example:

$ docker container run -d \
  --name web1 \
  --publish 8080:8080 \
  test:latest

Open a web browser and navigate to the DNS name or IP address of the Docker host that you are running the container from, and point it to port 8080. You will see the following web page.

If you are following along with Docker for Windows or Docker for Mac, you will be able to use localhost:8080 or 127.0.0.1:8080. If you’re following along on Play With Docker, you will be able to click the 8080 hyperlink above the terminal screen.

Figure 4.1
Figure 4.1

Windows example:

> docker container run -d \
  --name web1 \
  --publish 8080:80 \
  test:latest

Open a web browser and navigate to the DNS name or IP address of the Docker host that you are running the container from, and point it to port 8080. You will see the following web page.

The same rules apply if you’re following along with Docker Desktop or Play With Docker.

Figure 4.2
Figure 4.2

Well done. You’ve taken some application code from a remote Git repo and built it into a Docker image. You then ran a container from it. We call this “containerizing an app”.

Chapter Summary

In the Ops section of the chapter you downloaded a Docker image, launched a container from it, logged into the container, executed a command inside of it, and then stopped and deleted the container.

In the Dev section, you containerized a simple application by pulling some source code from GitHub and building it into an image using instructions in a Dockerfile. You then ran the containerized app.

This big picture view should help you with the up-coming chapters where we will dig deeper into images and containers.

About the Author
  • Nigel Poulton

    Nigel Poulton is a cloud-native subject matter expert who spends his life creating books and training videos on the latest cloud technologies. He is the author of best-selling books on Docker and Kubernetes and the most popular online training videos on the same topic. He is a Docker Captain. Prior to this, Nigel has held various infrastructure roles for large enterprises. When he is not playing with technology, he is dreaming about it. When he is not dreaming about it, he is reading and watching sci-fi. He wishes he lived in the future so he could explore spacetime, the universe, and tons of other mind-blowing stuff. He likes cars, football (soccer), and food. He has a fabulous wife and three children.

    Browse publications by this author
Latest Reviews (1 reviews total)
Very good service, I can view the document I purchased when I am not connected to the internet.
Docker Deep Dive
Unlock this book and the full library FREE for 7 days
Start now