Jenkins Continuous Integration Cookbook

By Alan Mark Berg
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Maintaining Jenkins

About this book

Jenkins is a highly popular continuous integration server. Its correct use supports a quality software development process. Jenkins is great at finding issues in software early and communicating it to a wide audience. Jenkins is also easily extendable with a simple framework for writing plugins. Currently there are over 400 plugins available for inclusion.

Jenkins Continuous Integration Cookbook has over 80 recipes describing practical ways to use Jenkins and expanding its feature set by selective use of the best of breed plugins. Jenkins has a simple framework for writing plugins. There are over 400 plugins available. Therefore, it is easy to get lost in possibilities. Using practical recipes, this book will guide you through the complexities. The recipes are bundled into themes including security, maintainability, communication, building software, the valid use of code metrics, testing remotely, and writing your first plugin.

Jenkins Continuous Integration Cookbook includes problem solving and how to do recipes for many common and less common tasks. A wide range of topics are covered from integrating, securing, and maintaining Jenkins in your organization to writing your first extension.

The book begins with common maintenance tasks followed by securing Jenkins and enabling SSO. Then it explores the relationship between Jenkins builds and the Maven pom.xml. You will then learn ways to effectively communicate with various target audiences (developers, project managers, the public). You will then explore source code metrics with related recipes, and set up and run remote stress and functional tests. The book finally lists a series of 11 interesting plugins with a concluding recipe on how to create your first plugin.

Jenkins Continuous Integration Cookbook describes solutions and optimizations to problems commonly found.

Publication date:
June 2012


Chapter 1. Maintaining Jenkins

This chapter provides recipes that help you to maintain the health of your Jenkins server.

In this chapter, we will cover the following recipes:

  • Using a sacrificial Jenkins instance

  • Backing up and restoring

  • Modifying the Jenkins configuration from the command line

  • Reporting about the overall disc usage

  • Deliberately failing builds through log parsing

  • A Job to warn about the disc usage violations

  • Keeping in contact with Jenkins through Firefox

  • Monitoring through JavaMelody

  • Keeping a track of the script glue

  • Scripting the Jenkins command-line interface

  • Global modifications of Jobs with Groovy

  • Signaling the need to archive



Jenkins is feature-rich and is vastly extendable through plugins. Jenkins talks with numerous external systems, and its Jobs work with many diverse technologies. Maintaining Jenkins in a rich environment is a challenge. Proper maintenance lowers the risk of failures, a few of which are listed as follows:

  • New plugins causing exceptions: There are a lot of good plugins being written with a rapid version change. In this situation, it is easy for you to accidentally add new versions of the plugins with new defects. There have been a number of times when the plugin suddenly stopped working, while it was being upgraded. To combat the risk of plugin exceptions, consider using a sacrificial Jenkins instance before releasing to a critical system.

  • Disks overflowing with artifacts: If you keep a build history, which includes artifacts such as war files, large sets of JAR files or other types of binaries, then your disk space is consumed at a surprising rate. Disc costs have decreased tremendously, but disk usage equates to longer backup times and more communication from the slave to the master. To minimize the risk of disk overflowing, you will need to consider your backup and restore policy and the associated build retention policy expressed in the advanced options of Jobs.

  • Script spaghetti: As Jobs are written by various development teams, the location and style of the included scripts vary. This makes it difficult for you to keep track. Consider using well-defined locations for your scripts, and a scripts repository managed through a plugin.

  • Resource depletion: As memory is consumed, or the number of intense Jobs increase, Jenkins slows down. Proper monitoring and quick reaction reduce their impact.

  • A general lack of consistency between Jobs due to organic growth: Jenkins is easy to install and use. The ability to seamlessly turn on plugins is addictive. The pace of adoption of Jenkins within an organization can be breathtaking. Without a consistent policy, your teams will introduce lots of plugins and lots of ways of performing the same work. Conventions improve the consistency and readability of Jobs and thus decrease the maintenance.

The recipes in this chapter are designed to address the risks mentioned. They represent only one set of approaches. If you have comments or improvements, feel free to contact me through my Packt Publishing e-mail address, or it would even be better if you still add tutorials to the Jenkins community wiki.


Signing up to the community

To add community bug reports, or to modify wiki pages, you will need to create an account at the following URL:


Using a sacrificial Jenkins instance

Continuous Integration (CI) servers are critical in the creation of deterministic release cycles. Any long-term instability in the CI server will reflect in the milestones of your project plans. Incremental upgrading is addictive and mostly straightforward, but should be seen in the light of the Jenkins wider role.

Before the release of plugins into the world of your main development cycle, it is worth aggressively deploying to a sacrificial Jenkins instance, and then sitting back and letting the system run the Jobs. This gives you enough time to react to any minor defects found.

There are many ways to set up a sacrificial instance. One is to use a virtual image of Ubuntu and share the workspace with the Host server (the server that the virtual machine runs on). There are a number of advantages to this approach:

  • Saving state: At any moment, you can save the state of the running virtual image and return to that running state later. This is excellent for short-term experiments that have a high risk of failure.

  • Ability to share images: You can run your virtual image anywhere that a player can run. This may include your home desktop or a hard core server.

  • Use a number of different operating systems: Good for node machines running integration tests or functional tests with multiple browser types.

  • Swap workspaces: By having the workspace outside the virtual image, you can test different version levels of the OS against one workspace. You can also test one version of Jenkins against different workspaces, with different plugin combinations.


The long-term support release

The community manages support of the enterprise through the release of a long-term supported version of Jenkins. This stable release version is older than the newest version and thus misses out on some of the newer features. You can download it from:

This recipe details the use of VirtualBox (, an open source virtual image player with a guest Debian OS image. The virtual image will mount a directory on the host server. You will then point Jenkins to the mounted directory. When the guest OS is restarted, it will automatically run against the shared directory.


Throughout the rest of this book, recipes will be cited using Ubuntu as the example OS.

Getting ready

You will need to download and install VirtualBox. You can find the detailed instructions at To download and unpack a Ubuntu virtual image, you can refer to the following URL:

Note that the newer images will be available at the time of reading. Feel free to try the most modern version; it is probable that the recipes might still work.


Security considerations

If you consider using other OS images a bad security practice, then you should create a Ubuntu image from a boot CD as mentioned at:

How to do it...

  1. Run VirtualBox and click on the New icon in the top-left hand corner. You will now see a wizard for installing virtual images.

  2. On the Welcome screen, click on the Next button.

  3. Set the Name to Jenkins_Ubuntu_11.04. The OS Type will be automatically updated. Click on the Next button.

  4. Set Memory to 2048 MB, and click on Next.

  5. Select Use existing hard disk. Browse and select the unpacked VDI image by clicking on the folder icon.

  6. Press the Create button.

  7. Start the virtual image by clicking on the Start icon.

  8. Log in to the guest OS with username and password as Ubuntu reverse.

  9. Change the password of user Ubuntu from a terminal.

  10. Install the Jenkins repository, as explained at

  11. Update the OS in case of security patches (this may take some time depending on bandwidth):

    apt-get update
    apt-get upgrade
  12. Install the kernel dkms module.

    sudo apt-get install dkms
  13. Install Jenkins.

    sudo apt-get install jenkins
  14. Install the kernel modules for VirtualBox.

    /etc/init.d/vboxadd setup
  15. Select Install Guest additions using the Devices menu option.

  16. Add the Jenkins user to the vboxsf group.

    sudo gedit /etc/group
  17. Modify the JENKINS_HOME variable in /etc/default/Jenkins to point to the mounted shared directory:

    sudo gedit /etc/default/Jenkins
  18. On the host OS, create the directory workspacej.

  19. Within the VirtualBox, right-click on the Ubuntu image, selecting properties.

  20. Update the Folder Path to point to the directory that you have previously created. In the following screenshot, the folder was created under my home directory.

  21. Restart VirtualBox, and start the Ubuntu Guest OS.

  22. On the Guest OS, run a web browser, and visit http://localhost:8080. You will see a locally running instance of Jenkins, ready for your experiments.

How it works...

Your recipe first installs a virtual image of Ubuntu, changes the password so that it is harder for others to log in, and updates the guest OS for security patches.

The Jenkins repository is added to the list of known repositories in the Guest OS. This involved locally installing a repository key. The key is used to verify that the packages, which are automatically downloaded, belong to a repository that you have agreed to trust. Once the trust is enabled, you can install through standard package management the most current version of Jenkins, and aggressively update it later.

You need to install the additional code called guest additions so that VirtualBox can share folders from the host. Guest additions depend on Dynamic Kernel Module Support (DKMS). DKMS allows bits of code to be dynamically added to the kernel. When you ran the command /etc/init.d/vboxadd setup, VirtualBox added guest addition modules through DKMS.


Warning: If you forget to add the DKMS module, then sharing folders will fail without any visual warning.

The default Jenkins instance now needs a little reconfiguration:

  • The jenkins user needs to belong to the the vboxsf group to have permission to use the shared folder

  • The /etc/init.d/jenkins startup script points to /etc/default/jenkins and picks up the values of specific properties, such as JENKINS_HOME.

Next, you added a shared folder to the guest OS from the VirtualBox GUI, and finally you had restarted VirtualBox and the guest OS to guarantee that the system was in a fully-configured and correctly initialized state.

There are a number of options for configuring VirtualBox with networking. You can find a good introductory text at:

There's more...

Two excellent sources of virtual images are:

See also

  • Monitoring through JavaMelody


Backing up and restoring

A core task for the smooth running of Jenkins is the scheduled backing up of its workspace. Not necessarily backing up all the artifacts, but at least the Jenkins configuration and the testing history are recorded by individual plugins.

Backups are not interesting unless you can restore. There are a wide range of stories on this subject. My favorite (and I won't name the well-known company involved) is the one in which somewhere in the early 70S, a company brought a very expensive piece of software and a tape backup facility to back up all the marketing results being harvested through their mainframes. However, not everything was automated. Every night, a tape needed to be moved into a specific slot. A poorly paid worker was allocated time. For a year, the worker would professionally fulfill the task. One day, a failure occurred and a backup was required. The backup failed to restore. The reason was that the worker also needed to press the record button every night, but this was not part of the tasks assigned to him. There was a failure to regularly test the restore process. The process failed, not the poorly paid person. Hence, learning the lessons of history, this recipe describes both backup and restore.

Currently, there is more than one plugin for backups. I have chosen the thinBackup plugin ( as it allows for scheduling.


The rapid evolution of plugins, and the validity of recipes

Plugins improve aggressively, and you may need to update them weekly. However, it is unlikely that the core configuration changes. But, it's quite likely that extra options will be added, increasing the variables that you input in the GUI. Therefore, the screen grabs shown in this book may be slightly different from the most modern version, but the recipes should remain intact.

Getting ready

Create a directory with read, write permissions for Jenkins, and install the ThinBackup plugin.


Murphy as a friend

You should assume the worst for all of the recipes in this book: aliens attacking, coffee on motherboard, cat eats cable, cable eats cat. Please make sure that you are using a sacrificial Jenkins instance.

How to do it...

  1. Click on the ThinBackup link in the Manage Jenkins page.

  2. Click on the link to Settings by the Toolset icon.

  3. Add the details as shown in the next screenshot. Here, /data/Jenkins/backups is a placeholder for the directory that you have previously created.

  4. Click on Save.

  5. Click on the Backup now icon.

  6. From the command line, visit your backup directory. You should now see an extra sub-directory named FULL-{timestamp}, where {timestamp} is the time to the second that the full backup was created.

  7. Click on the Restore icon.

  8. A select box restore backup form will be shown with the dates of the backups. Select the backup just created. Click on the Restore button.

  9. To guarantee the consistency, restart the Jenkins server.

How it works...

The backup scheduler uses the cron notation ( 1 0 * * 7 means every seventh day of the week at 00:01 AM. 1 1 * * * implies that differential backup occurs once per day at 1.01 A.M. Every seventh day, the previous differentials are deleted.

Differential backups contain only files that have been modified since the last full backup. The plugin looks at the last modified date to work out which files need to be backed up. The process can sometimes go wrong if another process changes the last modified date, without actually changing the content of the files.

61 is the number of directories created with backups. As we are cleaning up the differentials through the option Clean up differential backups, we will get to around 54 full backups, roughly a year of archives before cleaning up the oldest.

Backup build results were selected, as we assume that we are doing the cleaning within the Job. Under these conditions, the build results should not take much space. However, in case of misconfiguration, you should monitor the archive for disc usage.

Cleaning up differential backups saves you doing the clean-up work by hand.

Moving old backups to ZIP files saves space, but might temporarily slow down your Jenkins server.

Restore is a question of returning to the restore menu and choosing the date. I can't repeat this enough; you should practice a restore occasionally to avoid embarrassment.


Full backups are the safest as they restore to a known state. Therefore, don't generate too many differential backups between full backups; that's a false economy.

There's more...

Here are a couple more points for you to think about.

Checking for permission errors

If there are permission issues, the plugin fails silently. To discover these types of issues, you will need to check the Jenkins log file /var/log/jenkins/jenkins.log, for *NIX distributions and for log-level SEVERE. For example:

SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path {0}.

Testing exclude patterns

The following Perl script will allow you to test the exclude pattern. Simply replace the $content value with your Jenkins workspace location, and the $exclude_pattern with the pattern you wish to test. The script will print a list of the excluded files.

use File::Find;
my $content = "/var/lib/jenkins";
my $exclude_pattern = '^.*\.(war)|(class)|(jar)$';
find( \&excluded_file_summary, $content );
sub excluded_file_summary {
	if ((-f $File::Find::name)&&( $File::Find::name =~/$exclude_pattern/)){
            print "$File::Find::name\n";


Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at If you purchased this book elsewhere, you can visit and register to have the files e-mailed directly to you.

You can find the documentation for the standard Perl module file at:

For every file and directory under the location mentioned in $content, the line find(\&excluded_file_summary,$content); calls the function excluded_file_summary.

The exclude pattern'^.*\.(war)|(class)|(jar)$' ignores all war, class, and jar files.



If you are a Java Developer who occasionally writes Perl scripts, then consider using the EPIC plugin for Eclipse (

See also

  • Reporting about the overall disc usage

  • A Job to warn about the disc usage violations


Modifying Jenkins configuration from the command line

You may be wondering about the XML files at the top level of the Jenkins workspace. These are configuration files. config.xml is the main one, dealing with the default server values, but there are also specific ones for any plugins that have values set through the GUI.

There is also a jobs sub-directory underneath the workspace. Each individual Job configuration is contained in a sub-directory with the same name as the Job. The Job-specific configuration is then stored in config.xml within the sub-directory. There is a similar situation for the user's directory with one sub-directory per user, with the user information stored in its own config.xml file.

Under a controlled situation, where all the Jenkins servers in your infrastructure have the same plugins and version levels, it is possible for you to test on one sacrificial machine and then push the configuration files to all the other machines. You can then restart the servers with the Command-Line Interface (CLI).

This recipe familiarizes you with the main XML configuration structure and provides hints about the plugin API, based on the details of the XML.

Getting ready

You will need a fresh install of Jenkins with security enabled.

How to do it...

  1. In the top-level directory of Jenkins, look for the file named config.xml. Go to the line that has the<numExecutor> tag, and edit it by changing the number from 2 to 3, as follows:

  2. Restart the server. You will see that the number of executors has increased from a default of 2 to 3.

  3. Look for the file named thinBackup.xml. You will not find it unless you have installed the thinBackup plugin.

  4. Replay the recipe Back up and Restoring, and look again. You will now find the following XML file.

    <?xml version='1.0' encoding='UTF-8'?>
      <fullBackupSchedule>1 0 * * 7</fullBackupSchedule>
      <diffBackupSchedule>1 1 * * *</diffBackupSchedule>

How it works...

Jenkins uses Xstream ( to persist its configuration into a readable XML format. The XML files in the workspace are configuration files for plugins, tasks, and an assortment of other persisted information. config.xml is the main configuration file. Security settings and global configuration are set here and reflect changes made through the GUI. Plugins use the same structure, and the XML values correspond to member values in underlying plugin classes. The GUI itself is created from XML through the Jelly framework (

By restarting the server, you should be certain that any configuration changes are picked up during the initialization phase.

There's more...

Here are a few things for you to consider.

Turning off security

When you are testing new security features, it is easy to lock yourself out of Jenkins. You will not be able to log in again. To get around this problem, modify useSecurity to false in config.xml, and restart Jenkins. The security features are now turned off.

Finding JavaDoc for custom plugin extensions

The following line of code is the first line of the thin plugin configuration file thinBackup.xml, mentioning the class from which the information is persisted. The class name is a great Google search term. Plugins can extend the functionality of Jenkins, and there may be useful methods exposed for administrative Groovy scripts.


The effects of adding garbage

Jenkins is great at recognizing rubbish configuration as long as it is recognizable as a valid XML fragment. For example, add the following line of code to config.xml:

<garbage>yeuch blllllllaaaaaa</garbage>

When you reload the configuration, you will see the following error at the top of the manage Jenkins screen:

Pressing the Manage button will return you to a detailed page of the debug information, including the opportunity to reconcile the data.

From this, you can notice that Jenkins is developer-friendly when reading corrupted configuration that it does not understand.

See also

  • Using a sacrificial Jenkins instance

  • Participating in the community - Maven archetypes and plugins, Chapter 7, Exploring Plugins


Reporting overall disc usage

Organizations have their own way of dealing with increasing disc usage. Policy ranges from no policy, depending on ad-hoc human interactions, to the most state of the art software with central reporting facilities. Most organizations sit between these two extremes with mostly ad-hoc intervention, with some automatic reporting for the more crucial systems. With minimal effort, you can make Jenkins report disc usage from the GUI, and periodically run Groovy scripts that trigger helpful events.

This recipe highlights the disk usage plugin and uses the recipe as a vehicle to discuss the cost of keeping archives stored within the Jenkins workspace.

The disc usage plugin is the strongest in combination with an early warning system that notifies you when soft or hard disc limits are reached. The recipe: A Job to warn of disc usage violations through log parsing details a solution. Both the recipes show that configuring Jenkins requires little effort. Each step might even seem trivial. The power of Jenkins is that you can build complex responses out of a series of simple steps and scripts.

Getting ready

You will need to install the disc usage plugin.

How to do it...

  1. Press the Disk usage link under the Manage Jenkins page.

  2. After clicking on the Disk Usage link, Jenkins displays a page with each project and the builds and Workspace disc usage summary. Click on the top of the table to sort the workspace by file usage.

How it works...

Adding a plugin in Jenkins is very simple. The question is what are you going to do with the information.

It is easy for you to forget a tick box in a build, perhaps an advanced option is enabled where it should not be. Advanced options can at times be problematic, as they are not displayed directly in the GUI. You will need to hit the advanced button first, before reviewing. On a Friday afternoon, this might be one step too far.

Advanced options include artifact retention choices, which you will need to correctly configure to avoid overwhelming disc usage. In the previous example, the workspace for the Sakai CLE is 2GB. The size is to do with the Job having its own local Maven repository, as defined by the advanced option Use a private Maven repository. The option is easy for you to miss. In this case, there is nothing to be done, as trunk pulls in snapshot JARs, which might cause instability for other projects.

The simple act of being able to sort disc usage points the offending Jobs out to you, ready for further inspection of their advanced configuration.

There's more...

If you are keeping a large set of artifacts, it is an indicator of a failure of purpose of your use of Jenkins. Jenkins is the engine that pushes a product through its life cycle. For example, when a job builds snapshots every day, then you should be pushing the snapshots out to where developers find them most useful. That is not Jenkins but a Maven repository or a repository manager such as Artifactory (, Apache Archiva ( or Nexus ( These repository managers have significant advantages over dumping to disc. They have the following advantages:

  • Speed builds by acting as a cache: Development teams tend to work on similar or the same code. If you build and use the repository manager as a mirror, then the repository manager will cache the dependencies, and when Job Y asks for the same artifact, the download will be local.

  • Acts as a mechanism to share snapshots locally: Perhaps some of your snapshots are only for local consumption. The repository manager has facilities to limit the access.

  • GUI interface for ease of artifact management: All the three repository managers have intuitive GUIs, making your management tasks as easy as possible.

With these considerations in mind, if you are seeing a build-up of artifacts in Jenkins, where they are less accessible and beneficial than deployed to a repository, consider this a signal for the need to upgrade your infrastructure.

For further reading, see:


Retention policy

Jenkins can be a significant consumer of disk space. In the Job configuration, you can decide to either keep artifacts or remove them automatically after a given period of time. The issue with removing artifacts is that you will also remove the results from any automatic testing. Luckily, there is a simple trick for you to avoid this. When configuring a Job, click on Discard Old Builds, and then the Advanced checkbox, define the Max # of builds to keep with the artifacts. The artifacts are then removed after the number of builds specified, but the logs and results are kept. This has one important consequence; you have now allowed the reporting plugins to keep displaying a history of tests even though you have removed the other more disc consuming artifacts.

See also

  • Backing up and restoring


Deliberately failing builds through log parsing

Scenario: You have been asked to clean up the code removing depreciated Java methods across all the source contained under a Jenkins Jobs; that is a lot of code. If you miss some residue defects, then you will want the Jenkins build to fail.

What you need is a flexible log parser that can fail or warn about issues found in the build output. To the rescue: This recipe describes how you can configure a log parsing plugin that spots unwanted patterns in the console output and fails Jobs.

Getting ready

You will need to install the Log Parser Plugin as mentioned at:

How to do it...

  1. Create the log_rules directory owned by Jenkins, under the Jenkins workspace.

  2. Add the file named depreciated.rule to the log_rules directory with one line:

    error /DEPRECATED/
  3. Create a Job with a source code that gives deprecated warnings on compilation. In the following example, you are using the Roster tool from the Sakai project:

  4. Run the build. It should not fail.

  5. As shown in the next screenshot, visit the Manage configuration page for Jenkins, and to the Console Output section, add a description and location of the parsing rules file that was mentioned in step 2.

  6. Check the Console output (build log) parsing box in the Post-build Actions section of your Job.

  7. Check the Mark build Failed on Error checkbox.

  8. Select Kill Deprecated from the Select Parsing Rules list box.

  9. Build the Job; it should now fail.

  10. Click on the Parsed Console Output link in the left-hand menu. You will now be able to see the parsed errors.

How it works...

The global configuration page allows you to add files, each with a set of parsing rules. The rules use regular expressions mentioned in the home page of the plugin (

The rule file you used is composed of one line: error /DEPRECATED/.

If the pattern DEPRECATED (a case-sensitive test) is found in the console output, then the plugin considers this as an error, and the build fails. More lines to test can be added to the file. The first rule found wins. Other levels include warn and ok.

The source code pulled in from Sakai ( contains deprecated method and triggers the pattern.

The rules file has the distinct .rules extension in case you want to write an exclude rule during backups.

Once the plugin is installed, you can choose a Job between the rule files previously created.


This plugin empowers you to periodically scan for the obvious link and adapt to the new circumstances. You should consider sweeping systematically through a series of rule files failing suspect builds, until a full clean-up to in-house style has taken place.

There's more...

Two other examples of the common log patterns that are an issue, but do not normally fail a build are:

  • MD5 check sums: If a Maven repository has an artifact, but not its associated MD5 checksum file, then the build will download the artifact even if it already has a copy. Luckily, the process will leave a warning in the console output.

  • Failure to start up custom integration services: These failures might be logged at the warn or info level when you really want them to fail the build.

See also

  • A Job to warn about the disc usage violations through log parsing


A Job to warn about the disc usage violations through log parsing

The disk usage plugin is unlikely to fulfill all of your disc maintenance requirements. This recipe will show how you can strengthen disc monitoring by adding a custom Perl script to warn about the disc usage violations.

The script will generate two alerts: a hard error when the disc usage is above an acceptable level, and a softer warning when the disc is getting near to that limit. The log parser plugin will then react appropriately.


Using Perl is typical for a Jenkins Job, as Jenkins plays well and adapts to most environments. You can expect Perl, Bash, Ant, Maven, and a full range of scripts and binding code to be used in the battle to get the work done.

Getting ready

If you have not already done so, create a directory owned by Jenkins under the Jenkins workspace named log_rules. Also, make sure that the Perl scripting language is installed on your computer and is accessible by Jenkins. Perl is installed by default on Linux distributions. Activestate provides a decent Perl distribution for MAC and Windows (

How to do it...

  1. Add a file to the log_rules directory named disc.rule with the following two lines:

    error /HARD_LIMIT/
    warn /SOFT_LIMIT/
  2. Visit the Manage configuration page for Jenkins, and add a description as DISC_USAGE to the Console Output section. Point to the location of the Parsing Rules file.

  3. Add the following Perl script to a location of choice named, making sure that the Jenkins user can read the file.

    use File::Find;
    my $content = "/var/lib/jenkins";
    if ($#ARGV != 1 ) {
      print "[MISCONFIG ERROR] usage: hard soft (in Bytes)\n";
    my $total_bytes=0;
    my $hard_limit=$ARGV[0];
    my $soft_limit=$ARGV[1];
    find( \&size_summary, $content );
    if ($total_bytes >= $hard_limit){
      print "[HARD_LIMIT ERROR] $total_bytes >= $hard_limit (Bytes)\n";
    }elsif ($total_bytes >= $soft_limit){
      print "[SOFT_LIMIT WARN] $total_bytes >= $soft_limit (Bytes)\n";
      print "[SUCCESS] total bytes = $total_bytes\n";
    sub size_summary {
      if (-f $File::Find::name){
        $total_bytes+= -s $File::Find::name;
  4. Modify the $content variable to point to the Jenkins workspace.

  5. Create a free-style software project Job.

  6. Under the build section, add the build Step / Execute Shell. For the command, add perl 9000000 2000000.

  7. Feel free to change the hard and soft limits (9000000 2000000).

  8. Check the Console output (build log) parsing in the Post-build Actions section.

  9. Check the Mark build Unstable on Warning checkbox.

  10. Check the Mark build Failed on Error checkbox.

  11. Select DISC_USAGE from the Select Parsing Rules combo box.

  12. Run the build a number of times.

  13. Under build history on the left-hand, select the trend link. You can now view trend reports and see a timeline of success and failure.

How it works...

The Perl script expects two command-line inputs: hard and soft limits. The hard limit is the value in bytes that the disc utilization under the $content directory should not exceed. The soft limit is a smaller value in bytes that triggers a warning rather than an error. The warning gives the administrators time to clean up before the hard limit is reached.

The Perl script transverses the Jenkins workspace and counts the size of all the files. The script calls the method size_summary for each file or directory underneath the workspace.

If the hard limit is less than the content size, then the script generates the log output [HARD_LIMIT ERROR]. The parsing rules will pick this up and fail the build. If the soft limit is reached, then the script will generate the output [SOFT_LIMIT WARN]. The plugin will spot this due to the rule warn /SOFT_LIMIT/, and then signal a Job warn.

There's more...

Welcome to the wonderful world of Jenkins. You can now utilize all of the installed features at your disposal. The Job can be scheduled, e-mails can be sent out on failure. You can also tweet, add entries to Google calendar, trigger extra events, for example disc-cleaning builds, and so on. You are mostly limited by your imagination and 21st century technologies.

See also

  • Backing up and restoring


Keeping in contact with Jenkins through Firefox

If you are a Jenkins administrator, then it is your role to keep an eye on the ebb and flow of the build activity within your infrastructure. Builds can occasionally freeze or break due to non-coding reasons. If a build fails, and this is related to infrastructural issues, then you will need to be warned quickly. Jenkins can do this in numerous ways. Chapter 5, Communicating Through Jenkins is dedicated to the different approaches for different audiences. From e-mail, Twitter, and speaking servers, you can choose a wide range of prods, kicks, shouts, and pings. I could even imagine a Google summer of code project with a remotely controlled buggy moving to the sleeping administrator and then toting.

This recipe is one of the more pleasant ways for you to be reached. You will pull in the Jenkins RSS feeds using a Firefox add-on. This allows you to view the build process, while going about your everyday business.

Getting ready

You will need Firefox 5 or later installed on your computer and an account on at least one Jenkins instance, with a history of running Jobs.


A plug for the developers

If you like the add-on and want more features in the future, then it is enlightened in the self-interest to donate a few bucks at the add-on author's website.

How to do it...

  1. Select the Firefox tab at the top-left hand side of the browser.

  2. In the Search box (top-right) with the title Search all add-ons, search for Jenkins.

  3. Click on the Install button for the Jenkins Build monitor.

  4. Restart Firefox.

  5. Select the Firefox tab at the top left-hand side of the browser.

  6. Enable the Add-On Bar by selecting Options, and then Add-On Bar. Now, at the bottom right-hand side of Firefox, you will see a small Jenkins icon.

  7. Right-click on the icon.

  8. Select the preferences, and the Feeds screen appears.

  9. Add a recognizable, but short, name for your Jenkins instance. For example, Plugin test server.

  10. Add a URL using the following structure for Feed URL: http://host:port/rssAll e.g.: http://localhost:8080/rssAll.

  11. Check Enable executor monitoring.

  12. Click on the OK button. An area in the Add-On tool bar will appear with the name Plugin test Server of the Feed URL(s) displayed, and a health icon. If you hover your mouse over the name, then a more detailed status information will be displayed.

How it works...

Jenkins provides RSS feeds to make its status information accessible to a wide variety of tools. The Firefox add-on polls the configured feed and displays the information in a digestible format.

To configure for a specific crucial Job, you will need to use the following structure: http://host:port/job/job name/rssAll.

To view only the build failures, replace rssAll with rssFailed. To view only the last build, replace rssAll with rssLatest.

There's more...

If security is enabled on your Jenkins instances, then most of your RSS feeds will be password-protected. To add a password, you will need to modify the Feed URL to the following structure:

http://username:[email protected]:port/path



The negative aspect of using this add-on is that any Feed URL password is displayed in plain text during editing.

See also

Chapter 5, Communicating Through Jenkins:

  • Visualizing schedules the — Google calendar

  • Shouting at developers through Twitter


Monitoring through JavaMelody

JavaMelody ( is an open source project that provides comprehensive monitoring. The Jenkins plugin monitors both the Master instance of Jenkins and also its nodes. The plugin provides a detailed wealth of the important information. You can view the evolution charts ranging from a day or weeks to months of the main quantities, such as the CPU or the memory. Evolution charts are very good at pinpointing the scheduled Jobs that are resource-hungry. JavaMelody allows you to keep a pulse on the incremental degradation of resources. It eases the writing of reports by exporting statistics in a PDF format. Containing over 25 years of human effort, JavaMelody is feature-rich.

This recipe shows you how easy it is to install a JavaMelody plugin ( and discusses the troubleshooting strategies and their relationship with the generated metrics.


Community partnership

If you find this plugin useful, consider contributing back to either the plugin or the core JavaMelody project.

Getting ready

You will need to have installed the JavaMelody plugin.

How to do it...

  1. Click on the Monitoring Hudson/Jenkins master link on the Manage Jenkins page. You will now see the detailed monitoring information.

  2. Read the online help at the URL http://host:port/monitoring?resource=help/help.html, where the host and port point to your server.

  3. Review the monitoring of the node processes directly, by visiting http://host:port/monitoring/nodes.

How it works...

JavaMelody has the advantage of running as the Jenkins user and can gain access to all the relevant metrics. Its main disadvantage is that it runs as part of the server and will stop monitoring as soon as there is a failure. Because of this disadvantage, you should consider JavaMelody as part of the monitoring solution and not the whole.

There's more...

Monitoring is the foundation for comprehensive testing and troubleshooting. This section explores the relationship between these issues and the measurements exposed in the plugin.

Troubleshooting with JavaMelody - memory

Your Jenkins server can at times have memory issues due to greedy builds, leaky plugins, or some hidden complexity in the infrastructure.

JavaMelody has a comprehensive range of memory measurements, including a heap dump and a memory histogram.

The Java virtual machine divides the memory into various areas, and to clean up, it removes objects that have no references to other objects. Garbage collection can be CPU-intensive when it is busy, and the nearer you get to full memory, the busier the garbage collection becomes. To an external monitoring agent ,this looks like a CPU spike that is often difficult to track down. Just because the garbage collector manages memory, it is also a fallacy to believe that there is no potential for memory leakage in Java. Memory can be held too long by many common practices, such as custom caches or calls to native libraries.

Slow-burning memory leaks will show up as gentle slopes on the memory-related evolution graphs. If you suspect that you have a memory leak, then you can get the plugin to force a full garbage collection through the link Execute the garbage collector. If it is not a memory leak, then the gentle slope will abruptly fall.

Memory issues can also express themselves as large CPU spikes as the garbage collector frantically tries to clean up, but can barely clean enough space. The garbage collector can also pause the application while comprehensively looking for no longer referenced objects, and cause large response times for web browser requests. This can be seen through the mean and max times under the Statistics labeled http - 1 day.

Troubleshooting with JavaMelody - painful Jobs

You should consider the following points:

  • Offload work: For a stable infrastructure, offload as much work as possible from the master instance. If you have scheduled the tasks, keep the heaviest ones separate in time. Time separation not only evens out load, but also makes finding the problematic build easier through the observation of the evolution charts of JavaMelody. Also consider spatial separation; if a given node or a labeled set of nodes show problematic issues, then start switching around machine location of Jobs, and view their individual performance characteristics through http://host:port/monitoring/nodes.

  • Hardware is cheap: Compared to paying for human hours, buying an extra 8GB is cheap.


    A common gotcha is to add the memory to the server, but forget to update the init scripts to allow Jenkins to use more memory.

  • Review the build scripts: Javadoc generation, custom Ant scripts can fork JVMs, and reserve memory are defined within their own configuration. Programming errors can also be the cause of the frustration. Don't forget to review JavaMelody's report on the Statistic system error log and Statistic http system errors.

  • Don't forget external factors: Factors include backups, cron Jobs, updating the locate database, and network maintenance. These will show up as periodic patterns in the evolution charts.

  • Strength in numbers: Use the JavaMelody in combination with the disk usage plugin and others to keep a comprehensive overview of the vital statistics. Each plugin is simple to configure, but their usefulness to you will grow quicker than the maintenance costs of adding extra plugins.

See also

Chapter 7, Testing Remotely:

  • Running a script to obtain the monitoring information


Keeping a track of the script glue

There are negative implications for backing up and especially restoring if maintenance scripts are scattered across the infrastructure. It is better to keep your scripts in one place, and then run them remotely through the nodes. Consider placing your scripts under the Master Jenkins home directory. It would be even better for the community if you can share the less-sensitive scripts online. Your organization can reap the benefits; the scripts will then get some significant peer review and improvements.

In this recipe, we explore the use of the Scriptler plugin to manage your scripts locally and download useful scripts from an online catalog.

Getting ready

You will need to install the Scriptler plugin (

How to do it...

  1. Click on the Scriptler link under the Manage Jenkins page. You will notice the text in bold: Currently you do not have any scripts available, you can import scripts from a remote catalog or create your own.

  2. Click on the link on the left-hand side of Remote Script catalogs.

  3. Click on the icon of the floppy disk for getThreadDump. If the script is not available, then choose another script of your choice.

  4. You have now returned to the Scriptler main page. You will see three icons. Choose the furthest right to execute the script.

  5. You are now in the Run a script page. Select a node and hit the Run button.


    If the script fails with a message startup failed, then please add a new line between entry.key and for, and the script will then function correctly.

  6. To write a new Groovy script or to upload the one that you have on your local system, click on the Add a new Script link on the left-hand side.

How it works...

This plugin allows you to easily manage your Groovy scripts, and enforces a standard place for all Jenkins administrators to keep their code, making it easier for you to plan backups and indirectly share knowledge.

The plugin creates a directory named scriptler under the Jenkins workspace and persists the meta information about the files that you have created in the scriptler.xml file. A second file, scriptlerweb-catalog.xml, mentions the list of online files that you can download.

All the local scripts are contained in the sub-directory scripts.

There's more...

If enough people use this plugin, then the list of online scripts will radically increase the process of generating a significant library of reusable code. Therefore, if you have interesting Groovy scripts, then upload them. You will need to create a new account the first time to log in at:

Uploading your scripts allows people to vote on them and to send you feedback. The free peer review can only improve your scripting skills and increase your recognition in a wider community.

See also

  • Scripting the Jenkins command-line interface

  • Global modifications of Jobs with Groovy

  • Scripting the global build reports


Scripting the Jenkins command-line interface

The Jenkins Command-Line Interface (CLI),, allows you to perform a number of maintenance tasks on remote servers. Tasks include moving the Jenkins instances on and offline, triggering builds and running Groovy scripts. This makes for easy scripting of the most common chores.

In this recipe, you will log on to a Jenkins instance and run a Groovy script that looks for files greater than a certain size, and log off. The script represents a typical maintenance task. You can imagine chaining a second script to the first, to remove the large files found.


At the time of writing this chapter, the interactive Groovy shell was not working from the CLI. This is mentioned in the bug report:

Getting ready

Download the CLI JAR file from http://host/jnlpJars/jenkins-cli.jar.

Add the following script to a directory under the control of Jenkins and call it large_files.groovy.

root = jenkins.model.Jenkins.instance.getRootDir()
count = 0
size =0
root.eachFileRecurse() { file ->  
        if (file.size() > maxsize) {
            println "Thinking about deleting: ${file.getPath()}"
            // do things to large files here
}  pace used ${size/(1024*1024)} MB Number of files ${count}"

How to do it...

  1. Run the next command from a terminal, replacing http://host with the real address of your server, for example http://localhost:8080.

    java -jar _jenkins-cli.jar -s http://host login --username username
  2. Input your password.

  3. Look at the online help:

    java -jar _jenkins-cli.jar -s http://host help
  4. Run the Groovy script. The command-line output will now mention all the oversize files.

    java -jar _jenkins-cli.jar -s http://host groovy look.groovy
  5. Log out.

    java -jar _jenkins-cli.jar -s http://host logout.

How it works...

The CLI allows you to work from the command line and perform standard tasks. Wrapping the CLI in a shell script, such as bash, allows you to script maintenance tasks a large number of Jenkins instances at the same time. This recipe performs a lot of drudgework. In this case, it reviews X thousand files for oversized artifacts, saving you time that you can better spend on more interesting tasks.

Before performing any commands, you need to first authenticate through the login command.

Reviewing the script root jenkins.model.Jenkins.instance.getRootDir() uses the Jenkins framework to obtain a file, which points to the Jenkins workspace.

The maximum file size is set to 32MB through maxsize=1024*1024*32.

The script visits every file under the Jenkins workspace, using the standard Groovy method root.eachFileRecurse(){ file ->.


You can find the current JavaDoc for Jenkins at:

There's more...

The authentication used in this recipe can be improved. You can add your SSH public key under http://localhost:8080/user/{username}/configure (where username is your username), by cutting and pasting into the SSH Public Keys section. You can find detailed instructions at:

At the time of writing, there were some issues with the key approach. See Feel free to resort back to the method used in this recipe, which has proven to work stably, though less securely.


The CLI is easily-extendable, and therefore over time, the CLI's command list increases. It is therefore important that you occasionally check the in-built help.

See also

  • Global modifications of Jobs with Groovy

  • Scripting global build reports


Global modifications of Jobs with Groovy

Jenkins is not only a continuous integration server but also a rich framework with an exposed internal structure available from within the script console. You can programmatically iterate through the Jobs, plugins, node configuration, and a variety of rich objects. As the number of Jobs increase, you will notice that scripting becomes more valuable. For example, imagine that you need to increase custom memory settings across 100 Jobs. A Groovy script can do that in seconds.

This recipe is a representative example: You will run a script that iterates through all Jobs. The script then finds one specific Job by its name, and then updates the description of that Job with a random number.

Getting ready

Log in to Jenkins with an administrative account.

How to do it...

  1. Create an empty Job named MyTest.

  2. Within the Manage Jenkins page, click on the Script console link.

  3. Cut and paste the following script into the text area input.

    import java.util.Random 
    Random random = new Random()  
    hudson.model.Hudson.instance.items.each { job ->
      println ("Class: ${job.class}") 
      println ("Name: ${}")
      println ("Root Dir: ${job.rootDir}")
      println ("URL: ${job.url}")
      println ("Absolute URL: ${job.absoluteUrl}") 
      if ("MyTest".equals({
        println ("Description: ${job.description}")
        job.setDescription("This is a test id: ${random.nextInt(99999999)}")
  4. Click on the run button. The results should be similar to the following screenshot:

  5. Run the script a second time, and you will notice that the random number in the description has now changed.

  6. Copy and run the following script:

    for (slave in hudson.model.Hudson.instance.slaves) {
      println "Slave class: ${slave.class}"
      println "Slave name: ${}" 
      println "Slave URL: ${slave.rootPath}"
      println "Slave URL: ${slave.labelString}\n"
  7. If you have no slave instances on your Jenkins master, then no results are returned. Otherwise, the output will look similar to the following screenshot:

How it works...

Jenkins has a rich framework, which is exposed to the script console. The first script iterates through Jobs whose parent is AbstractItem ( The second script iterates through instances of slave objects (

There's more...

For the hardcore Java developer: If you don't know how to do a programmatic task, then an excellent source of example code is the Jenkins subversion directories for plugins (


If you are interested in donating your own plugin, review the information at:

See also

  • Scripting the Jenkins command-line interface

  • Scripting the global build reports


Signaling the need to archive

Each development team is unique. Teams have their own way of doing business. In many organizations, there are one-off tasks that need to be done periodically, for example at the end of each year.

This recipe details a script that checks for the last successful run of any Job, and if the year is different to the current year, then a warning is set at the beginning of the Jobs description. Thus, hinting to you it is time to perform some action, such as archiving and then deleting. You can, of course, programmatically do the archiving. However, for high value actions, it is worth forcing interceding, letting the Groovy scripts focus your attention.

Getting ready

Log in to Jenkins with an administrative account.

How to do it...

Within the Manage Jenkins page, click on the Script console link, and run the following script:

import hudson.model.Run;
import java.text.DateFormat;

def warning='<font color=\'red\'>[ARCHIVE]</font> '
def now=new Date()

for (job in hudson.model.Hudson.instance.items) { 
    println "\nName: ${}"
    Run lastSuccessfulBuild = job.getLastSuccessfulBuild()
    if (lastSuccessfulBuild != null) {
      def time = lastSuccessfulBuild.getTimestamp().getTime()
      if (now.year.equals(time.year)){
         println("Project has same year as build");
      }else {
        if (job.description.startsWith(warning)){
            println("Description has already been changed");

Any project that had its last successful build in another year than this will have the word [ARCHIVE] in red, added at the start of its description.

How it works...

Reviewing the code listing:

A warning string is defined, and the current date is stored in now. Each Job in Jenkins is programmatically iterated through the for statement.

Jenkins has a class to store information about the running of builds. The runtime information is retrieved through job.getLastSuccessfulBuild(), and is stored in the lastSuccessfulBuild instance. If no successful build has occurred, then lastSuccessfulBuild is set to null, otherwise it has the runtime information.

The time of the last successful build is retrieved, and then stored in the time instance through lastSuccessfulBuild.getTimestamp().getTime().

The current year is compared with the year of the last successful build, and if they are different and the warning string has not already been added to the front of the Job description, then the description is updated.



You will find the Job API mentioned at and the Run information at

There's more...

Before writing your own code, you should review what already exists. With 300 plugins, Jenkins has a large, freely-available, and openly licensed example code base. Although in this case the standard API was used, it is well worth reviewing the plugin code base. In this example, you will find part of the code re-used from the lastsuccessversioncolumn plugin (


If you find any defects while reviewing the plugin code base, please contribute to the community through patches and bug reports.

See also

  • Scripting the Jenkins command-line interface

  • Global modifications of Jobs with Groovy

About the Author

  • Alan Mark Berg

    Alan Mark Berg, BSc, MSc, PGCE, has been the lead developer at Central Computer Services at the University of Amsterdam since 1998. He is currently working in an Innovation Work Group that accelerates the creation of new and exciting services. In his famously scarce spare time, he writes. Alan has a bachelor's degree, two master's degrees, a teaching qualification, and quality assurance certifications. He has also coauthored two Packt Publishing books about Sakai , a highly successful open source learning management platform used by millions of students around the world. He has won a couple of awards, including the Sakai Fellowship and Teaching With Sakai Innovation Award (TWSIA). Alan enjoys working with talent; this forces him to improve his own competencies. This motivation is why Alan enjoys working in energetic, open source communities of interest. At the time of writing, he is on the board of directors of the Apereo Foundation and is the community officer for its Learning Analytics Initiative.In previous incarnations, Alan was a QA director, a technical writer, an Internet/Linux course writer, a product line development officer, and a teacher. He likes to get his hands dirty with building, gluing systems, exploring data, and turning it into actionable information. He remains agile by ruining various development and acceptance environments and generally rampaging through the green fields of technological opportunity.

    Browse publications by this author
Jenkins Continuous Integration Cookbook
Unlock this book and the full library for FREE
Start free trial