Chapter 7. Mesos Containerizers

This chapter briefly introduces the concepts of containers, and talks a bit about Docker, probably the most popular container technology available today. It also provides a detailed overview of the different containerizer options in Mesos besides introducing some other topics such as networking for Mesos-managed containers and fetcher cache. Finally, an example for deploying containerized apps in Mesos is provided for better understanding. The following topics will be covered in this chapter:

• Containers

• Docker

• Mesos Containerizer

  • Mesos Containerizer

  • Docker Containerizer

  • Composing Containerizer

• Networking for Mesos-managed containers

• Mesos Image Provisioning

• Fetcher Cache

• Deploying containerized apps using Docker and Mesos

Containers

A Linux Container, (referred to simply as container for the rest of this chapter) allows applications to run on an allocated share of resources within an isolated, individual environment. Since all containers share the Operating system (OS) of the host machine and do not require the OS to be loaded up, they can be created in a matter of seconds.

Container technology, based on operating system level virtualization, has been present for over a decade now. OS level virtualization is a method by which an OS kernel allows creation of many user namespace instances (also called containers) instead of only one.

We can look at containers as encapsulated, individually deployable components running as isolated instances on the same kernel. Containers have a big advantage over traditional technologies such as bare metal, meaning servers with an operating system or virtualized environments such as Microsoft Hyper-V. From a developer's point of view, we can just package our application and dependencies...

Docker

Docker is an open-source platform that automates the process by which any application is deployed in the form of a container that is portable, lightweight, self-sufficient, and which can run virtually anywhere. Primarily based on the LXC or Linux Container, Docker is used by developers and system administrators while working with distributed applications. Rather than being an underlying technology, this platform acts as a comprehensive abstraction layer that enables developers to package or containerize an application, including its dependencies, and run it on any infrastructure. Simply put, Docker containers function like shipping containers which offer a standard and reliable way of shipping literally any application.

Docker provides the Development and IT Operations teams with much needed agility and control to 'build, ship, and run any app, anywhere.'

• Build: Docker gives you the flexibility to create an application from micro-services, and not worry about probable inconsistencies...

Mesos containerizer

This is the default containerizer type provided by Mesos. In this type, tasks can be run through an array of pluggable isolators provided by Mesos. It can be enabled by configuring the agent flag as

--containerizers=mesos

This type is typically used when:

• User needs to control the task environment through Mesos without having to rely on other container solutions.

• Fine-grained OS controls are desired.

• Custom resource isolation needs to be added for tasks.

• User needs to control certain resource parameters (for example, disk usage limits) which are not exposed by other container solutions.

Any task which doesn't specify ContainerInfo::DockerInfo will be handled by the Mesos containerizer.

The launching process

The container launching process includes the following steps:

• Preparation of calls is done on every isolator.

• The launcher, responsible for forking/destroying containers, is used to fork the executor. The forked 'child' cannot execute until the isolation step is completed.

• The...

Networking for Mesos-managed containers

One of the major goals to provide networking-related support in Mesos was to develop a pluggable architecture leveraging which custom networking mechanisms could be implemented by users as per their requirements. Since networking requirements vary across different deployment scenarios (cloud, on-premise, private cloud, or other hybrid models), it is not practical to create a monolithic networking mechanism that caters to all needs. Mesos' pluggable architecture proves to be very useful in tackling this.

To provide networking support, many opt-in extensions were introduced in Mesos components from version 0.25.0 onwards. The opt-in structure allows existing frameworks without networking support to continue operating seamlessly on newer Mesos versions. Mesos enables integration with other networking mechanisms, and provides features like service discovery, IP per container, and isolation of tasks.

Networking support is provided through a Mesos module,...

Mesos Image Provisioner

An Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime.

Most of the container specifications, such as Docker, App Container (AppC), or Open Container Project (OCP), combine implementation and specification of the image format with other container components such as resource isolation and task execution to a large extent. The Mesos Image Provisioner seeks to enhance the capabilities of Mesos Containerizer by providing support for container filesystem provisioning from multiple image formats, while also providing features such as resource isolation by combining with other components such as Isolators.

Mesos containers created using the Mesos Containerizer are provided with a root filesystem already provisioned with Docker or AppC images using the Mesos Image Provisioner.

The container filesystem image is described by a new message Image, which is given in the following code:

message Image...

Mesos fetcher

The Mesos fetcher is a way by which resources can be downloaded in the task sandbox directory while preparing the task execution. The framework requesting the execution of the task sends a list of CommandInfo::URI values as part of the TaskInfo message, which in turn serves as the Mesos fetcher input.

The Mesos fetcher natively supports the FTP and HTTP protocols, and is also able to copy over files from a filesystem. It also supports all Hadoop client protocols such as Amazon Simple Storage Service (S3), Hadoop distributed Filesystem (HDFS), and so on.

Every Uniform Resource Identifier (URI) that is requested gets downloaded, by default, directly into the sandbox directory. Multiple requests for the same URI results in copies of that resource being downloaded again and again. The downloaded URIs can also alternatively be cached in a specified directory for reuse.

Mechanism

The mechanism comprises the following:

1. Each slave consists of one internal fetcher instance which is leveraged...

Deploying containerized apps using Docker and Mesos

This section gives a brief overview of deploying a Docker containerized Node.js application on Mesos using Marathon. This requires you to have Docker and fig already installed on the machine. Let's follow the steps listed next to carry out the deployment:

1. Since we are deploying a simple Node.js application, we can start off by creating a simple App.js to print Hello World, a simple hello world Node.js program.

   var http = require('http'); 
   // Configure our HTTP server to respond with Hello World to all requests.
   var server = http.createServer(function (request, response) {
     response.writeHead(200, {"Content-Type": "text/plain"});
     response.end("Hello World	");
   }); 
   // Listen on port 8000, IP defaults to "0.0.0.0"
   server.listen(8000); 
   // Put a friendly message on the terminal
   console.log("Server running at http://127.0.0.1:8000/");

2. Next we create the package.json file with the following contents:

   {
     "name": "hello-world",
     "description"...

Summary

This chapter touched upon several important topics related to containerization in Mesos. Resource isolation is one of the most touted features of Mesos, and the topics explained in this chapter have hopefully helped you understand this feature.

In the next chapter, we will see some of the important Big Data frameworks which are currently supported by Mesos such as Hadoop, Spark, and Storm and understand how these can be set up and configured on Mesos.