Chapter 8. Using Docker Stack and Compose YAML Files to Deploy Swarm Services

Copy and paste is a design error.                            –David Parnas

The most common question I receive during my Docker-related talks and workshops is usually related to Swarm and Compose.

Someone: How can I use Docker Compose with Docker Swarm?

Me: You can't! You can convert your Compose files into a Bundle that does not support all Swarm features. If you want to use Swarm to its fullest, be prepared for docker service create commands that contain a never ending list of arguments.

Such an answer was usually followed with disappointment. Docker Compose showed us the advantages of specifying everything in a YAML file as opposed to trying to remember all the arguments we have to pass to docker commands. It allowed us to store service definitions in a repository thus providing a reproducible and well-documented process for managing them. Docker Compose replaced bash scripts, and we loved it. Then, Docker v1.12...

Swarm cluster setup

To setup an example Swarm cluster using Docker Machine, please run the commands that follow.

All the commands from this chapter are available in the 07-docker-stack.sh (https://gist.github.com/vfarcic/57422c77223d40e97320900fcf76a550) Gist:

cd cloud-provisioning

git pull

scripts/dm-swarm.sh

Now we're ready to deploy the docker-flow-proxy service.

Creating Swarm services through Docker stack commands

We'll start by creating a network:

Note

A note to Windows users You might experience a problem with volumes not being mapped correctly. If you see an Invalid volume specification error, please export the environment variable COMPOSE_CONVERT_WINDOWS_PATHS set to 0:export COMPOSE_CONVERT_WINDOWS_PATHS=0 Please make sure that the variable is exported before you run docker-compose or docker stack deploy.

eval $(docker-machine env swarm-1)

docker network create --driver overlay proxy

The proxy network will be dedicated to the proxy container and services that will be attached to it.

We'll use docker-compose-stack.yml (https://github.com/vfarcic/docker-flow-proxy/blob/master/docker-compose-stack.yml) from the vfarcic/docker-flow-proxy (https://github.com/vfarcic/docker-flow-proxy) repository to create docker-flow-proxy and docker-flow-swarm-listener services.

The content of the docker-compose-stack.yml file is as follows:

version: "3"

services:...

Deploying more stacks

Let's deploy another stack.

This time we'll use Docker stack defined in the Compose file docker-compose-stack.yml (https://github.com/vfarcic/go-demo/blob/master/docker-compose-stack.yml) located in the vfarcic/go-demo (https://github.com/vfarcic/go-demo/) repository. It is as follows:

version: '3'

services:

  main:
    image: vfarcic/go-demo
    environment:
      -DB=db
    networks:
      - proxy
      - default
    deploy:
      replicas: 3
      labels:
        - com.df.notify=true
        - com.df.distribute=true
        - com.df.servicePath=/demo
        - com.df.port=8080

  db:
    image: mongo
    networks:
      - default

networks:
  default:
    external: false
  proxy:
    external: true

The stack defines two services (main and db). They will communicate with each other through the default network that will be created automatically by the stack (no need for docker network create command). Since the main service is an API, it should be accessible through...

To stack or not to stack

Docker stack is a great addition to the Swarm Mode. We do not have to deal with docker service create commands that tend to have a never ending list of arguments. With services specified in Compose YAML files, we can replace those long commands with a simple docker stack deploy. If those YAML files are stored in code repositories, we can apply the same practices to service deployments as to any other area of software engineering. We can track changes, do code reviews, share with others, and so on.

The addition of the Docker stack command and its ability to use Compose files is a very welcome addition to the Docker ecosystem.

Throughout the rest of the book, we'll use docker service create commands when exploring new services and docker stack deploy to create those we are already familiar with. If you have trouble converting docker service create commands into stacks, please take a look at the vfarcic/docker-flow-stacks (https://github.com/vfarcic/docker-flow-stacks...

Cleanup

Please remove Docker Machine VMs we created. You might need those resources for some other tasks:

exit

docker-machine rm -f swarm-1 swarm-2 swarm-3