Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Apache Mesos Cookbook
Apache Mesos Cookbook

Apache Mesos Cookbook: Efficiently handle and manage tasks in a distributed environment

Arrow left icon
Profile Icon David Blomquist Profile Icon Tomasz Janiszewski
Arrow right icon
£15.99 £22.99
Book Aug 2017 146 pages 1st Edition
eBook
£15.99 £22.99
Print
£28.99
Subscription
Free Trial
Renews at £16.99p/m
Arrow left icon
Profile Icon David Blomquist Profile Icon Tomasz Janiszewski
Arrow right icon
£15.99 £22.99
Book Aug 2017 146 pages 1st Edition
eBook
£15.99 £22.99
Print
£28.99
Subscription
Free Trial
Renews at £16.99p/m
eBook
£15.99 £22.99
Print
£28.99
Subscription
Free Trial
Renews at £16.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Table of content icon View table of contents Preview book icon Preview Book

Apache Mesos Cookbook

Chapter 1. Getting Started with Apache Mesos

In this chapter, we present an overview of the Mesos architecture and recipes for installing Mesos on Linux and Mac. The following are the recipes covered in this chapter:

  • Installing Mesos on Ubuntu 16.04 from packages
  • Installing Mesos on Ubuntu 14.04 from packages
  • Installing Mesos on CentOS 7 and RHEL 7 from packages
  • Preparing Ubuntu 16.04 for a Mesos installation from source code
  • Preparing Ubuntu 14.04 for a Mesos installation from source code
  • Preparing OS X (Yosemite and El Capitan) for a Mesos installation from source code
  • Downloading, building, and installing the Mesos source code

Introduction


Apache Mesos is cluster management software that distributes the combined resources of many individual servers to applications through frameworks. Mesos is open source software that is free to download and use in accordance with the Apache License 2.0. This chapter will provide the reader with recipes for installing and configuring Apache Mesos.

Mesos can run on Linux, Mac, and Windows. However, we recommend running Mesos on Linux for production deployments and on Mac and Windows for development purposes only. Mesos can be installed from source code or from binary packages downloaded from repositories. We will cover a few installation methods on select operating systems in this chapter. The reasons for covering these specific operating systems and installation methods are as follows:

  • The operating systems natively include a kernel that supports full resource isolation
  • The operating systems are current as of this writing with long-term support
  • The operating systems and installation methods do not require workarounds or an excessive number of external repositories
  • The installation methods use the latest stable version of Mesos, whether it is from binary packages or source code

Based on these criteria, we have covered the following installation methods for this book:

  • Mesosphere packages on Ubuntu 16.04 LTS
  • Mesosphere packages on Ubuntu 14.04 LTS
  • Mesosphere packages on CentOS 7 and RHEL 7
  • Source code on Ubuntu 16.04 LTS
  • Source code on Ubuntu 14.04 LTS
  • Source code on OS X (Yosemite and El Capitan)

Mesosphere, a company founded by one of the original developers of Mesos, provides free, open source binary packages as well as commercial support for Mesos. Mesosphere binary packages are well maintained and provide an easy way to install and run Mesos. We use Mesosphere packages installed on Ubuntu 14.04 as the base environment for the recipes in chapters 3 - 8 and we recommend you use that installation method with this book. If you need to run Mesos one of the other supported operating systems, we provide you with the installation instructions in this chapter but you will need to adapt the recipes in chapters 3 - 8 for use with your operating system. Installing Mesos from source will allow you to customize the build and install process and enable and disable features so we will also guide you through a source install in this chapter. If you want a development environment on a Mac, building from source on OS X is the only way to go.

We will guide you through the installations in the following sections but first, you will need to plan your Mesos deployment. For a Mesos development environment, you only need one host or node. The node can be a physical computer, a virtual machine, or a cloud instance. For a production cluster, we recommend at least three master nodes and as many slave nodes as you will need to support your application frameworks. You can think of the slave nodes as a pool of CPU, RAM, and storage that can be increased by simply adding more slave nodes. Mesos makes it very easy to add slave nodes to an existing cluster as your application requirements grow. At this point, you should determine whether you will be building a Mesos development environment or a production cluster and you should have an idea of how many master and slave nodes you will need. The next sections will provide recipes for installing Mesos in the environment of your choice.

Installing Mesos on Ubuntu 16.04 from packages


In this recipe, we will be installing Mesos .deb packages from the Mesosphere repositories using apt.

Getting ready

You must be running a 64-bit version of the Ubuntu 16.04 operating system and it should be patched to the most current patch level using apt-get prior to installing the Mesos packages.

How to do it...

  1. First, download and install the OpenPGP key for the Mesosphere packages:
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
  1. Now install the Mesosphere repository:
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')$ CODENAME=$(lsb_release -cs)$ echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main"| 
sudo tee /etc/apt/sources.list.d/mesosphere.list
  1. Update the apt-get package indexes:
$ sudo apt-get update
  1. Install Mesos and the included ZooKeeper binaries:
$ sudo apt-get -y install mesos
  1. At this point, you can start Mesos to do some basic testing. To start the Mesos master and agent (slave) daemons, execute the following:
$ sudo service mesos-master start$ sudo service mesos-slave start
  1. To validate the Mesos installation, open a browser and point it to http://<ipaddress>:5050. Replace <ipaddress> with the actual address of the host with the new Mesos installation.

How it works...

The Mesosphere packages provide the software required to run Mesos. Next, you will configure ZooKeeper, which is covered in Chapter 2, Implementing High Availability with Apache ZooKeeper.

See also

If you prefer to build and install Mesos on Ubuntu from source code, we will cover that in an upcoming section in this chapter.

Installing Mesos on Ubuntu 14.04 from packages


In this recipe, we will be installing Mesos .deb packages from the Mesosphere repositories using apt.

Getting ready

You must be running a 64-bit version of the Ubuntu 14.04 operating system and it should be patched to the most current patch level using apt-get prior to installing the Mesos packages.

How to do it...

  1. First, download and install the OpenPGP key for the Mesosphere packages:
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
  1. Now install the Mesosphere repository:
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')$ CODENAME=$(lsb_release -cs)$ echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main"|
sudo tee /etc/apt/sources.list.d/mesosphere.list
  1. Update the apt-get package indexes:
$ sudo apt-get update
  1. Install Mesos and the included ZooKeeper binaries:
$ sudo apt-get -y install mesos
  1. At this point, you can start Mesos to do some basic testing. To start the Mesos master and agent (slave) daemons, execute the following command:
$ sudo service mesos-master start$ sudo service mesos-slave start
  1. To validate the Mesos installation, open a browser and point it to http://<ipaddress>:5050. Replace <ipaddress> with the actual address of the host with the new Mesos installation.

How it works...

The Mesosphere packages provide the software required to run Mesos. Next, you will configure ZooKeeper, which is covered in Chapter 2, Implementing High Availability with Apache ZooKeeper.

See also

If you prefer to build and install Mesos on Ubuntu from source code, we will cover that in an upcoming section in this chapter.

Installing Mesos on CentOS 7 and RHEL 7 from packages


In this recipe, we will be installing Mesos .rpm packages from the Mesosphere repositories using yum.

Getting ready

Your CentOS 7 or RHEL 7 operating system should be patched to the most current patch level using yum prior to installing the Mesosphere packages.

How to do it...

  1. First, add the Mesosphere repository:
$ sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm  
  1. And now, install Mesos and ZooKeeper:
$ sudo yum -y install mesos mesosphere-zookeeper
  1. At this point, you can start Mesos to do some basic testing. To start the Mesos master and agent (slave) daemons, execute the following:
$ sudo service mesos-master start$ sudo service mesos-slave start
  1. To validate the Mesos installation, open a browser and point it to http://<ipaddress>:5050. Replace <ipaddress> with the actual address of the host with the new Mesos installation.

How it works...

The Mesosphere packages provide the software required to run Mesos. Next, you will configure ZooKeeper, which is covered in Chapter 2, Implementing High Availability with Apache ZooKeeper.

See also

If you prefer to build and install Mesos from source code on RHEL 7 or CentOS 7, you can find installation instructions for CentOS 7 on the mesos.apache.org website. We do not cover installing Mesos source code on RHEL7 or CentOS 7 in this book due to dependencies that require packages from multiple third-party repositories.

Preparing Ubuntu 16.04 for a Mesos installation from source code


In this recipe, we will prepare the Ubuntu 16.04 operating system for a Mesos source code installation.

Getting ready

You must be running a 64-bit version of the Ubuntu 16.04 operating system and it should be patched to the most current patch level using apt-get prior to building the Mesos source code.

How to do it...

  1. First, we need to sync the latest package lists from the apt repositories with the update command:
$ sudo apt-get update
  1. Then we will install the prerequisite packages for building and running the Mesos source code:
$ sudo apt-get install -y tar wget git openjdk-8-
jdk autoconf libtool
build-essential python-dev python-boto libcurl4-
nss-dev libsasl2-dev
libsasl2-modules maven libapr1-dev libsvn-dev 
libghc-zlib-dev
  1. Next, continue to the Downloading, building, and installing the Mesos source code recipe at the end of this chapter.

How it works...

Preparing the operating system will enable us to build and install the Mesos source code. Next, you will need to download the Mesos source files and build the code.

Preparing Ubuntu 14.04 for a Mesos installation from source code


In this recipe, we will prepare the Ubuntu 14.04 operating system for a Mesos source code installation.

Getting ready

You must be running a 64-bit version of the Ubuntu 14.04 operating system and it should be patched to the most current patch level using apt-get prior to building the Mesos source code.

How to do it...

  1. First, we need to sync the latest package lists from the apt repositories with the update command:
$ sudo apt-get update
  1. Then, we install the prerequisite packages for building and running the Mesos source code:
$ sudo apt-get install -y tar wget git openjdk-7-
jdk autoconf libtool
build-essential python-dev python-boto libcurl4-
nss-dev libsasl2-dev
libsasl2-modules maven libapr1-dev libsvn-dev
  1. Next, continue to the Downloading, building, and installing the Mesos source code recipe at the end of this chapter.

How it works...

Preparing the operating system will enable us to build and install the Mesos source code. Next, you will need to download the Mesos source files and build the code.

Preparing OS X (Yosemite and El Capitan) for a Mesos Installation from source code


In this recipe, we will prepare the OS X operating system for a Mesos source code installation.

Getting ready

Building the current version of Mesos (which is 1.0.1 at the time of this writing) requires GCC 4.8.1+ or Clang 3.5+. You should be running a 64-bit version of OS X and it should be patched to the most current patch level prior to building the Mesos source code.

How to do it...

  1. Install command-line tools, Homebrew, Java, and libraries required for Mesos:
$ xcode-select --install
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew install Caskroom/cask/java
$ brew install wget git autoconf automake libtool subversion maven
  1. Next, continue to the Downloading, building, and installing the Mesos source code recipe at the end of this chapter.

How it works...

Preparing the operating system will enable us to build and install the Mesos source code. Next, you will need to download the Mesos source files and build the code.

Downloading, building, and installing the Mesos source code


In this recipe, we will download the Mesos source files, build the code, and install the Mesos binaries and libraries.

Getting ready

Follow the previous sections in this chapter to prepare your operating system for a Mesos source installation.

How to do it...

  1. Browse to the following location: http://mesos.apache.org/downloads/.
  2. Download the TAR file for the most recent stable release of Mesos. For example, to download Mesos version 1.0.1:
$ wget http://www.apache.org/dist/mesos/1.0.1/mesos-1.0.1.tar.gz
  1. Extract the contents of the TAR file. For example, to extract the Mesos 1.0.1 TAR file:
$ tar xvzf mesos-1.0.1.tar.gz
  1. Change the directory to the extracted TAR file directory created in the previous step, for example:
$ cd mesos-1.0.1
  1. Create the build directory:
$ mkdir build
  1. Change directory to the build directory:
$ cd build
  1. Configure the Mesos build script. You can run ../configure --help to see all the available configuration options. Refer to the There's more... section at the end of this recipe for more configuration tips:
$ ../configure
  1. Build the Mesos binaries (executing 'make -j <number of cores> V=0 will decrease both the build time and log verbosity on multicore systems). Refer to the There's more... section at the end of this recipe for more build tips:
$ make
  1. Optional: Run some tests:
$ make check
  1. Optional: Execute the make install command if you would like to install the Mesos binaries and libraries in /usr/local/ and add them to the system path. Mesos can also be installed in other locations or run directly from the build directory. Refer to the There's more... section at the end of this recipe for instructions on installing Mesos to alternative locations, as well as for other installation tips:
$ sudo make install
  1. Optional: For Ubuntu only, create links to the Mesos libraries if you chose the default installation to /usr/local:
$ sudo ldconfig
  1. At this point, you can start Mesos to do some basic testing. To start the Mesos master and agent daemons, first create the mesos working directory:
$ sudo mkdir /var/lib/mesos
  1. Change into the Mesos source build directory:
$ cd build
  1. Now start the Mesos master:
$ sudo ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos
  1. And start the Mesos agent (in a separate terminal session):
$ sudo ./bin/mesos-agent.sh --master=127.0.0.1:5050 --work_dir=/var/lib/mesos
  1. To validate the Mesos installation, open a browser on the Mesos host and point it to http://127.0.0.1:5050. If you wish to validate with a browser remotely, you will need to replace 127.0.0.1 with the real IP of the Mesos host in the previous commands and restart both the master and agent. You will also need to use the real IP of the Mesos host in the browser.

Note

Warning When using the real IP in the command line, the Mesos master and agent processes will fail to start unless the Mesos host is registered in DNS.

How it works...

Building the Mesos source code will create the binaries needed to run Mesos. Next, you will configure ZooKeeper, which is covered in Chapter 2, Implementing High Availability with Apache ZooKeeper.

There's more...

Refer to the following tips.

Configuration tips

There are a number of options that can be configured in the Mesos build script. For example, if you want to build Mesos with SSL support, you will need to enable the ssl and libevent features by executing ../configure --enable-ssl --enable-libevent prior to executing make. To see a full list of the configure options, execute ../configure --help from the Mesos build directory.

Build tips

If you are building out a Mesos cluster using hosts with the same operating system version, you can configure and build the source code on one server and then copy that source directory to the other servers. You can then skip the ../configure and make commands and just run Mesos from the build directory or execute make install to install it without having to go through the build (make) process on every server.

If you do plan on copying pre-built Mesos software to other servers, make sure all of the servers have the supporting packages required to run Mesos and that the OS is patched to the same version.

Installation tips

The default behavior of make install is to install the Mesos binaries and libraries in the /usr/local path. If you would like to change the installation path, execute ../configure --prefix=/path/to/install/directory during the configure stage prior to the build (make) and install (make install) stages.

If at some point you would like to uninstall the Mesos binaries from the installation path, you can execute make uninstall from the source code build directory. After building and installing the Mesos source code, we recommend archiving the source code directory for future reference.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Learn to install and configure Mesos to suit the needs of your organization
  • Follow step-by-step instructions to deploy application frameworks on top of Mesos, saving you many hours of research and trial and error
  • Use this practical guide packed with powerful recipes to implement Mesos and easily integrate it with other application frameworks

Description

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks. The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper. Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We’ll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch. Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.

What you will learn

  • [*] Set up Mesos on different operating systems
  • [*] Use the Marathon and Chronos frameworks to manage multiple applications
  • [*] Work with Mesos and Docker
  • [*] Integrate Mesos with Spark and other big data frameworks
  • [*] Use networking features in Mesos for effective communication between containers
  • [*] Configure Mesos for high availability using Zookeeper
  • [*] Secure your Mesos clusters with SASL and Authorization ACLs
  • [*] Solve everyday problems and discover the best practices

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Aug 2, 2017
Length 146 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781785884627
Vendor :
Apache
Category :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want

Product Details

Publication date : Aug 2, 2017
Length 146 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781785884627
Vendor :
Apache
Category :

Packt Subscriptions

See our plans and pricing
Modal Close icon
£16.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
£169.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts
£234.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts

Frequently bought together

Stars icon
Total £ 40.98 59.98 19.00 saved
Mastering Mesos
£24.99 £36.99
Apache Mesos Cookbook
£15.99 £22.99
=
Book stack Total £ 40.98 59.98 19.00 saved Stars icon

Table of Contents

15 Chapters
Title Page Chevron down icon Chevron up icon
Credits Chevron down icon Chevron up icon
About the Authors Chevron down icon Chevron up icon
About the Reviewer Chevron down icon Chevron up icon
www.PacktPub.com Chevron down icon Chevron up icon
Customer Feedback Chevron down icon Chevron up icon
Preface Chevron down icon Chevron up icon
1. Getting Started with Apache Mesos Chevron down icon Chevron up icon
2. Implementing High Availability with Apache ZooKeeper Chevron down icon Chevron up icon
3. Running and Maintaining Mesos Chevron down icon Chevron up icon
4. Understanding the Scheduler API Chevron down icon Chevron up icon
5. Managing Containers Chevron down icon Chevron up icon
6. Deploying PaaS with Marathon Chevron down icon Chevron up icon
7. Job Scheduling with Metronome Chevron down icon Chevron up icon
8. Continuous Integration with Jenkins Chevron down icon Chevron up icon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.