Chapter 8. Continuous Integration with Jenkins

In this chapter, you will learn how to install and configure Jenkins to use Mesos resources and run Jenkins jobs. This reduces infrastructure costs, since the same Mesos cluster could be used for Jenkins and other frameworks. Jenkins can scale dynamically, so developers aren't waiting for their builds. A pipeline introduced in Jenkins 2.0 and PaaS solutions such as Marathon allow you to build a whole system, taking care of building, testing and deployment. We will cover the following recipes in this chapter:

• Building the Jenkins Mesos plugin
• Configuring and running Jenkins jobs
• Running Jenkins jobs in Docker

Introduction

Jenkins is a well-known automation system. It is written in Java as a fork of Hudson (an automation system created by Sun Microsystems). Jenkins is designed to automate continuous integration and continuous deployment. It supports many code repositories with different triggers and build targets. It can be extended with plugins. The biggest advantage of Jenkins over other tools is its ability to run jobs on Mesos, which can reduce costs by reusing idle servers or easily spawning new build agents.

Building the Jenkins Mesos plugin

By default, Jenkins uses statically created agents and runs jobs on them. We can extend this behavior with a plugin that will make Jenkins use Mesos as a resource manager. Jenkins will register as a Mesos framework and accept offers when it needs to run a job.

Getting ready

You need to have Mesos up and running. See Chapter 1, Getting Started with Apache Mesos for more information.

How to do it...

The Jenkins Mesos plugin installation is a bit harder than Marathon. There are no official binary packages for it so it must be installed from sources:

1. First, we need to download the source code:

curl -L https://github.com/jenkinsci/mesos-plugin/archive/mesos-0.14.0.tar.gz | tar -zx
cd jenkinsci-mesos-plugin-*

2. The plugin is written in Java and to build it we need Maven (mvn):

sudo apt install maven

3. Finally, build the package:

mvn package

If everything goes smoothly, you should see information, that all tests passed and the plugin package will be placed in target/mesos.hpi...

Installing Jenkins

We will be using Jenkins 2.32.1. Please be aware that future versions of Jenkins may require different steps if the screen changes. You can skip this recipe if you already have a running Jenkins installation.

How to do it...

1. Installing Jenkins is pretty straightforward. First, we need to add the Jenkins repository and it's key to the package sources of our system. After refreshing the packages list, we should be able to install Jenkins:

wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

2. By default, Jenkins will run on port 8080 and it should be running and waiting for configuration.
3. To change the default port, open /etc/default/Jenkins, find the following lines, and change the port and restart service:

# port for HTTP connector (default 8080; disable with -1)
HTTP_PORT=8080

4. At this point, Jenkins...

Enabling the Jenkins Mesos plugin

In this recipe, you will learn how to enable the Mesos Jenkins plugin and configure a job to be run on Mesos.

Getting ready

We assume you have built the Mesos Jenkins plugin. You will also need the Mesos cluster.

How to do it...

1. The first step is to install the Mesos Jenkins plugin. To do so, navigate to the Plugin Manager by clicking Manage Jenkins | Manage Plugins, and select the Advanced tab. You should see the following screen:

2. Click Choose file and select the previously built plugin to upload it. Once the plugin is installed, you have to configure it. To do so, go to the configuration (Manage Jenkins | Configure System). At the bottom of the page, the cloud section should appear. Fill in all the fields with the desired configuration values:

3. If you now disable Advanced On- demand framework registration, you should see the Jenkins Scheduler registered in the Mesos frameworks.

Note

Remember to configure Slave username to the existing system user on Mesos agents...