Extending Jenkins

4.5 (2 reviews total)
By Donald Simpson
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Preparatory Steps

About this book

Jenkins CI is the leading open source continuous integration server. It is written in Java and has a wealth of plugins to support the building and testing of virtually any project. Jenkins supports multiple Software Configuration Management tools such as Git, Subversion, and Mercurial.

This book explores and explains the many extension points and customizations that Jenkins offers its users, and teaches you how to develop your own Jenkins extensions and plugins.

First, you will learn how to adapt Jenkins and leverage its abilities to empower DevOps, Continuous Integration, Continuous Deployment, and Agile projects. Next, you will find out how to reduce the cost of modern software development, increase the quality of deliveries, and thereby reduce the time to market. We will also teach you how to create your own custom plugins using Extension points.

Finally, we will show you how to combine everything you learned over the course of the book into one real-world scenario.

Publication date:
December 2015


Chapter 1. Preparatory Steps

In this first chapter, we will start off by looking at Jenkins from several different perspectives; how to obtain and run it, some of the ways and the reasons people use it, and what it provides to them. In doing so, we will take a look at some standard use cases and examine how a Jenkins installation will often evolve over a period of time—typically starting off with only the basic installation and core features, then progressively becoming more customized and advanced with different types of extensions. We will start off with "ready-made" plugins, and then progress towards extending these before looking at how to develop your own plugins.

We will then summarize the high-level aims of this book, and give the details of what you should hopefully gain from them.

We will provide an overview of the various tools and the environment setup that you will need in order to run the practical examples covered in the subsequent chapters, and we will review the best practices of Continuous Integration (CI) by identifying some of the ways that Jenkins can be used to achieve them.

Throughout this book, it is assumed that you already have some working knowledge of Jenkins, so we will not spend much time covering the basics, such as installing and starting Jenkins, or detailing the usage of standard features and core functions.

If you would like more details on these topics, there are numerous helpful tutorials and examples available online; the Use Jenkins section of the Jenkins homepage, https://jenkins-ci.org, is often a good starting point for help with general setup and usage questions.


Getting started with Jenkins

As a Java application, Jenkins can be installed and run in different ways depending on your requirements, personal preferences, and the environment that you are running it in.

The simplest and easiest approach to quickly get Jenkins up and running is by setting up Java, downloading the latest Jenkins WAR file from the Jenkins homepage (www.jenkins-ci.org), and then simply starting it from the command line like this:

java –jar jenkins.war

The following figure demonstrates the use of this approach by running just two simple commands:

  1. wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war:

    This command downloads the latest version of Jenkins from the main site.

    wget is a Linux utility that fetches files from the Web—if you are on a platform that does not have wget, you can simply save the link (the jenkins.war file) via your browser to a working directory instead.

    The URL is obtained by copying the Latest & Greatest link from the homepage at https://jenkins-ci.org/. Note that there is also an option to download and use the Long-Term Support release instead of the current, latest, and greatest, as explained here: https://wiki.jenkins-ci.org/display/JENKINS/LTS+Release+Line.

    This is preferable for more conservative installations, where stability is more important than having latest features.

  2. java –jar jenkins.war:

    This second command tells Java to run the WAR file that we just downloaded as an application, which produces the resulting output that you can see in the following screenshot—Jenkins unpacking from the WAR file, checking and initializing the various subsystems, and starting up a process on port 8080:

    Downloading and starting Jenkins

This simple process is usually all that is required to both download the latest version of Jenkins and get it up and running. You should now be able to access the web interface at http://localhost:8080 through your browser and begin setting up jobs to make Jenkins work for you:

The Jenkins start page


Extending the basic setup

When you exit from the command prompt or shell that started the process that we looked at previously, the Jenkins instance will stop with the exit, so for anything beyond a very quick ad hoc test, some form of initialization or process management script is highly recommended. Such a script can also be easily tailored to perform a few "nice to have" functions for you, for example, things such as these:

  • Starting up at system boot time

  • Catering to stop|start|restart|status requests

  • Redirecting console output to a log file so that you can monitor it for issues

  • Running as a background/daemon process

  • Running on a nonstandard port by setting the --httpPort= parameter, in cases where port 8080 is already used by another application

  • Binding to a specific network interface, rather than the default value using the --httpListenAddress= option

This Ubuntu-based example script from the home page demonstrates many of the previously mentioned features of Jenkins that is running under Tomcat. The script can be found at https://wiki.jenkins-ci.org/display/JENKINS/JenkinsLinuxStartupScript and is as follows:

# Startup script for the Jenkins Continuous Integration server
# (via Jakarta Tomcat Java Servlets and JSP server)
# chkconfig: - 85 15
# description: Jakarta Tomcat Java Servlets and JSP server
# processname: jenkins
# pidfile: /home/jenkins/jenkins-tomcat.pid

# Set Tomcat environment.
export PATH=/usr/local/bin:$PATH
export HOME=/home/jenkins
export JAVA_HOME=/usr/lib/jvm/java-6-sun
export JENKINS_BASEDIR=/home/jenkins
export TOMCAT_HOME=$JENKINS_BASEDIR/apache-tomcat-6.0.18
export CATALINA_PID=$JENKINS_BASEDIR/jenkins-tomcat.pid
export CATALINA_OPTS="-DJENKINS_HOME=$JENKINS_BASEDIR/jenkins-home -Xmx512m -Djava.awt.headless=true"

# Source function library.
. /etc/rc.d/init.d/functions

[ -f $TOMCAT_HOME/bin/catalina.sh ] || exit 0

export PATH=$PATH:/usr/bin:/usr/local/bin

# See how we were called.
case "$1" in
        # Start daemon.
        echo -n "Starting Tomcat: "
        su -p -s /bin/sh $JENKINS_USER -c "$TOMCAT_HOME/bin/catalina.sh start"
        [ $RETVAL = 0 ] && touch $LOCKFILE
        # Stop daemons.
        echo -n "Shutting down Tomcat: "
        su -p -s /bin/sh $JENKINS_USER -c "$TOMCAT_HOME/bin/catalina.sh stop"
        [ $RETVAL = 0 ] && rm -f $LOCKFILE
        $0 stop
        $0 start
       [ -e $LOCKFILE ] && $0 restart
        status -p $CATALINA_PID -l $(basename $LOCKFILE) jenkins
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
exit 0

Note that the http://jenkins-ci.org/ home page also hosts Native Installers for many popular operating systems under the Native packages column. These pages provide download links and installation instructions for each OS.

You may want to look at running Jenkins in a J2EE container too, which can often lead to a more seamless fit with your existing software stack and architecture. This may mean that you will inherit additional benefits, such as the container's logging, authentication, authorization, or resilience. Jenkins can be run with many popular J2EE compatible containers, including the following:

  • WebSphere

  • WebLogic

  • Tomcat

  • JBoss

  • Jetty

  • Jonas

There are more init script examples and detailed installation instructions readily available on the Web, which should cover any combination of operating system and container setup. The point of this is that you should be able to set up Jenkins to suit your environment and preferences.

For the purposes of this book, we will assume that Jenkins is being run directly from the command line on the local host. If you are using a J2EE container to host the application or running the application on a remote host, the only difference you will notice is that you may need to perform additional admin and deployment steps.


Jenkins evolution

Typically, most users or organizations will start off on their Jenkins journey by setting up a basic, standard Jenkins installation to manage a few simple development tasks. The most common use is to build your source code, either periodically or whenever it changes in your central repository (Git, Subversion, and so on).

Using Jenkins to automate this type of simple and repetitive task often provides a lot of useful benefits very quickly and easily. Straight out of the box, so to speak, you get a bundle of helpful features, such as task scheduling and job triggering, building and testing report pages, sending out email notifications and alerts when there are new issues, and providing rapid and live feedback of how healthy (or not!) your code base currently is. If you don't already have a tool in place to provide these things, then setting up a standard Jenkins instance will provide these initial basic features, which on their own may well transform your development process.

The next logical step after this is to gradually add a little more intelligence and complexity to the setup—does the code compile ok? How many unit tests have been passed now, how long does the application take to compile? Oh, and could we show on a web page who has changed which parts of the code base? Is our application running faster or better than it was previously, and is it stable? Even before we begin to add any type of extension or customization, the core Jenkins installation provides a plethora of options here—you can choose to build your application on any platform that runs Java (which means pretty much anywhere these days), and you can also do this in whatever way that suits you and your current setup the best, including using the standard and popular build tools such as Ant or Maven, and/or re-using your existing Ant or Maven build scripts, or your Linux Shell or Windows DOS scripts.

You can also easily set up a cross-platform environment by deploying Jenkins Slave Nodes, which will allow you to run different jobs on different hosts. This can be useful in the environments that use a combination of operating systems; for example, your application runs on Linux, and you want to run your browser-based tests using Internet Explorer on a Windows host.

This ability to act as an easily configurable "wrapper" for your existing process, combined with the flexible nature of Jenkins, makes it very easy to adapt your particular setup to suit your requirements with minimal change or interruption. This makes Jenkins far easier to implement than having to change your existing build and deployment processes and practices just to accommodate the requirements of a new tool.

After this stage, the benefits of setting up a Continuous Integration environment often become quite obvious: if we can automatically build our code and package our application so easily, wouldn't it be great if we could go on to deploy it too? And then, if we did that, we could automatically test how our new application performs on a replica of the target platform!

On reaching this point, Jenkins will be a pivotal tool in your Continuous Integration process, and the more you can extend it to suit your growing and specific requirements, the more benefit you will receive from it.

This leads us to extending Jenkins, which is what we will be looking at in the rest of the book.

The simplest way to extend Jenkins is through its fantastic and ever-expanding wealth of plugins. It is always recommended and informative to browse through them; existing plugins are frequently being improved upon and updated with new features, and new plugins are being added to the list all the time. We are going to do more than just review a few popular plugins here though—by the end of this book, you should have the ability to take your usage of Jenkins to the next level to create your own custom plugins and extensions and work with the many features and interfaces that Jenkins provides us with for extension and interaction.

We will be taking a detailed look at the following:

  • The different ways in which we can use the existing features

  • Interacting with Jenkins through its various interfaces and APIs

  • How to interact with Jenkins from within your IDE

  • Ways to build upon the existing functionality to suit your needs

  • Developing, testing, and building your own custom Jenkins extension

Here are the main tools that we will be using to help us extend Jenkins, along with some information on setting them up, and the sources for further help and information if required:

  • Java Development Kit (JDK): You will need a version of this at the same bit level as your Java IDE, that is, both will need to be 32 bit or 64 bit, depending on your architecture and preference. You can choose from IBM, Oracle, or OpenJDK 6.0 or later. Each vendor supplies installation instructions for all major platforms.

  • Java IDE: We will mainly be using Eclipse, but will cater to NetBeans and IntelliJ too, where possible.

    The most recent versions of each of these IDEs are available at their respective websites:

  • Mylyn: This is used to communicate with Jenkins from our IDE. If Mylyn is not already included in your IDE, you can download it from the Eclipse site here: http://www.eclipse.org/mylyn/downloads/. We will cover this in detail in Chapter 3, Jenkins and the IDE.

  • Maven: We will be using Maven 3 to build the Jenkins source code and our own custom plugin. Maven is a Java tool, so it will need to know about the JDK of your system.

  • Jenkins Source: This will be downloaded by Maven.

  • Git: On most Linux platforms, the equivalent of sudo apt-get install git should suffice. On Mac, there are several options, including the git-osx installer on Sourceforge. For Microsoft Windows, there is an executable installer available at http://msysgit.github.io/.

We will go in to more specifics on the installation and usage of each of these components as we use them in the later chapters.


Continuous Integration with Jenkins

Before we conclude this chapter, here is a list of the key practices of Continuous Integration (as defined by Martin Fowler in 2006) with the examples of the ways in which Jenkins can be used to help you achieve them:

  • Maintain a Single Source Repository: Jenkins can interact with all modern source code and version control repositories—some abilities are built-in, others can be added as extensions.

  • Automate the Build: As described earlier in the use cases, this is one of the core aims of Jenkins and often the main driver to start using Jenkins.

  • Make Your Build Self-Testing: This is usually the second step in setting up a CI environment with Jenkins—once you automate the building of the code, automating the tests as well is a natural progression.

  • Everyone Commits To the Mainline Every Day: We can't really force developers to do this, unfortunately. However, we can quite easily highlight and report who is doing—or not doing—what, which should eventually help them learn to follow this best practice.

  • Every Commit Should Build the Mainline on an Integration Machine: Builds can be triggered by developer commits, and Jenkins Slave Nodes can be used to build and provide accurate replica environments to build upon.

  • Fix Broken Builds Immediately: This is another developer best practice that needs to be adopted—when Jenkins shows red, the top focus should be on fixing the issue until it shows green. No one should commit new changes while the build is broken, and Jenkins can be configured to communicate the current status in the most effective way.

  • Keep the Build Fast: By offloading and spreading work to distributed Slave Nodes and by breaking down builds to identify and focus on the areas that have changed, Jenkins can be tuned to provide a rapid response to changes—a good target would be to check in a change and obtain a clear indication of the result or impact under 10 minutes.

  • Test in a Clone of the Production Environment: After compiling the new change, downstream Jenkins jobs can be created that will prepare the environment and take it to the required level—applying database changes, starting up dependent processes, and deploying other prerequisites. Using virtual machines or containers in conjunction with Jenkins to automatically start up environments in a known-good state can be very useful here.

  • Make it Easy for Anyone to Get the Latest Executable: Jenkins can be set up to act as a web server hosting the latest version at a known location so that everyone (and other processes/consumers) can easily fetch it, or it can also be used to send out details and links to interested parties whenever a new version has been uploaded to Nexus, Artifactory, and so on.

  • Everyone can see what's happening: There are many ways in which Jenkins communications can be extended—email alerts, desktop notifications, Information Radiators, RSS feeds, Instant Messaging, and many more—from lava lamps and traffic lights to the ubiquitous toy rocket launchers!

  • Automate Deployment: This is usually a logical progression of the Build -> Test -> Deploy automation sequence, and Jenkins can help in many ways; with Slave Nodes running on the deployment host, or jobs set up to connect to the target and update it with the most recently built artifact.

The benefits that can be realized once you have achieved the preceding best practices are often many and significant—your team will release software of higher quality and will do this more quickly and for less cost than before. However, setting up an automated build, test, and deployment pipeline will never be enough in itself; the tests, environment, and culture must be of sufficient quality too, and having the developers, managers, and business owners "buy in" to the processes and practices often makes all the difference.



In this preparatory chapter, we have taken a look at the basics of Jenkins; how it is used from both functional and practical points of view. We have run through a high-level overview of the toolset that we will be using to extend Jenkins in the following chapters and reviewed the best practices for Continuous Integration along with the ways in which Jenkins can be used to help your team achieve them.

In the next chapter, we will take a look at the ways in which we can extend the Jenkins user interface to make it more productive and intelligent, and how we can extend the user experience to make life easier and more productive for end users, as well as for Jenkins admins, build scripts, and processes.

About the Author

  • Donald Simpson

    Donald Simpson is an information technology consultant based in Scotland, UK.

    He specializes in helping organizations improve the quality and reduce the cost of software development through the adoption of process automation and Agile methodologies.

    Starting out as a Java developer, Donald's interest in application servers, networking, and automation led him to a career as a build engineer. He remains highly technical and hands-on and enjoys learning about new technologies and finding ways to automate and improve manual processes.

    He can be reached at www.donaldsimpson.co.uk.

    Browse publications by this author

Latest Reviews

(2 reviews total)
Decent book, topic I was interested in
Extending Jenkins
Unlock this book and the full library for $5 a month*
Start now