Switch to the store?

Jenkins Continuous Integration Cookbook - Second Edition

More Information
  • Integrate Jenkins with LDAP and SSO solutions
  • Maintain and secure Jenkins
  • Run an integration server firing automatic functional and performance tests
  • Communicate through social media and by plotting custom data
  • Skin Jenkins to your corporate look and feel
  • Refine the use of code metrics to improve quality
  • Write your first custom Jenkins plugin
  • Apply tweaks to optimize your use of Jenkins

Jenkins is an award-wining and one of the most popular Continuous Integration servers in the market today. It was designed to maintain, secure, communicate, test, build, and improve the software development process.

This book starts by examining the most common maintenance tasks. This is followed by steps that enable you to enhance the overall security of Jenkins. You will then explore the relationship between Jenkins builds and Maven pom.xml. Then, you will learn how to use plugins to display code metrics and fail builds to improve quality, followed by how to run performance and functional tests against a web application and web services. Finally, you will see what the available plugins are, concluding with best practices to improve quality.



Read an Extract from the book

Analyzing project data with the R plugin

This recipe describes how to use R to process metrics on each file in your project workspace. The recipe does this by traversing the workspace and collecting a list of files of a particular extension such as Java. The R script then analyzes each file individually and finally plots the results in a graphical format to a PDF file. The workflow is common to almost all quality-related analysis of software projects. This recipe is easily customized for tasks that are more complex.

In this example, we are looking at the size in words of the text files, printing to the console the names of large files and plotting the sizes of all files. From the visual representation, you can easily see which files are particularly large. If your property file is much larger than the other property files, it is probably corrupt. If a Java file is too large, it is difficult to read and understand.

How to do it...

  1. Create a free-style job with the name ch5.R.project.data.
  2. In the Source Code Management section, select Subversion.
  3. Add the Repository URL as https://source.sakaiproject.org/svn/profile2/trunk.
  4. In the build section, under Add build step, select Execute R script.
  5. In the Script text area add the following code:
    <- function(file){
      text <- readLines(file,encoding="UTF-8")
      if (length(text)> 500) print(file)
    javaFiles <-
    list.files(Sys.getenv('WORKSPACE'), recursive 
    = TRUE, full.names = TRUE, pattern = "\\.java$")
    <- list.files(Sys.getenv('WORKSPACE'), 
    recursive = TRUE, full.names = TRUE, pattern = "\\.properties$")
    resultJava <-
    sapply(javaFiles, processFile)
    <- sapply(propertiesFiles,processFile)
    <-paste('Lengths_JAVA_',Sys.getenv('BUILD_NUMBER'),'.pdf',sep ="")
    of length of JAVA files")
    filename <-
    of length of Property files")
  6. Click on the Save button.
  7. Click on the Build Now icon.
  8. Review the console output from the build. It should appear similar to the following:
    At revision 313948
    no change for 
     https://source.sakaiproject.org/svn/profile2/trunk since 
     the previous build
    [ch5.R.project.data] $ Rscript /tmp/hudson7641363251840585368.R
    [1] "/var/lib/jenkins/workspace/ch5.project.data/api/src/java/o
    14: In readLines(file, encoding = "UTF-8") :
      incomplete final line found on 
    Finished: SUCCESS
  9. Visit the workspace and review the files Lengths_Properties_1.pdf, Lengths_JAVA_1.pdf.
  10. Notice the straggler files with a large number of lines. Property files should be of roughly similar length, as they contain, in this case, the international translations for the GUI.

 This feels like a well-balanced project, as there are only a few files that have a large number of lines of code.

How it works...

You loaded in the profile2 tool from subversion https://source.sakaiproject.org/svn/profile2/trunk. This code is used by millions of students around the world and represents mature, realistic production code.

Within your R script, you defined a function that takes a filename as input and then reads the file into a text object. The function then checks to see whether the number of lines is greater than 500. If it is greater than 500 lines then the filename is printed to the console output. Finally, the function returns the number of lines in the text file.

processFile <- function(file){
  text <- readLines(file,encoding="UTF-8")
  if (length(text)> 500) print(file)

Next, the script discovers the property and Java files under the workspace. The file search is filtered by the value defined in the pattern argument. In this case, .java:

javaFiles <- list.files(Sys.getenv('WORKSPACE'), recursive = TRUE, 
 full.names = TRUE, pattern = "\\.java$")
propertiesFiles <- list.files(Sys.getenv('WORKSPACE'), recursive = 
 TRUE, full.names = TRUE, pattern = "\\.properties$")

The list of filenames is passed one name at a time to the processFile function you have previously defined. The results are a list of file lengths that are stored in the resultJava and resultProperties objects:

resultJava <- sapply(javaFiles, processFile)
resultProperties <- sapply(propertiesFiles,processFile)

The warnings() function produces a list of issues generated while running the sapply command:

14: In readLines(file, encoding = "UTF-8") :
  incomplete final line found on 

This is stating that a new line was expected at the end of the file. It is not a critical issue. Showing the warnings is a helpful approach to discovering corrupted files.

Finally, we generate two histograms of the results, one for the Java file and the other for the properties files. The filename is created from a constant string followed by the BUILD_NUMBER environment variable that is set uniquely for each build. The pdf function tells R that the output is to be stored in a PDF file and the hist function draws a histogram of the results:

filename <-paste('Lengths_JAVA_',Sys.getenv('BUILD_NUMBER'),'.pdf',sep="")
hist(resultJava,main="Frequency of length of JAVA files")

There's more...

When writing R code for processing your files, don't reinvent the wheel. R has many libraries for manipulating text. The stringi library is one example (http://cran.r-project.org/web/packages/stringi/stringi.pdf). Here is some example code that counts the number of words in a text file:

processFile <-function(file){
paste("Number of words in file:", results[4])

The script defines the function processFile. The function requires a filename. The file is read into the stri_stats_latex function. This function is included in the stringi library. It returns a summary of the file as a vector (a series of numbers).

The file.choose() function pops up a dialog that allows you to browse your file system and choose a file. The call returns the fully qualified path to the file. It passes the value to the processFile function call. The results are stored in the results vector. The script then prints out the fourth number that is the number of words in the file.

Another interesting R package for text mining is tm: (http://cran.r-project.org/web/packages/tm/tm.pdf). The tm package has the ability to load a set of text files and analyze them in many different ways.

  • Explore the use of more than 40 best-of-breed plug-ins for improving efficiency
  • Secure and maintain Jenkins by integrating it with LDAP and CAS, which is a Single Sign-on solution
  • Step-by-step, easy-to-use instructions to optimize the existing features of Jenkins using the complete set of plug-ins that Jenkins offers
Page Count 408
Course Length 12 hours 14 minutes
Date Of Publication 30 Jan 2015