Continuous Integration

Exclusive offer: get 50% off this eBook here
PHPUnit Essentials

PHPUnit Essentials — Save 50%

Get started with PHPUnit and learn how to write and test codes using advanced technologies with this book and ebook

$26.99    $13.50
by Zdenek Machek | May 2014 | Open Source Web Development

You have code and you have tests, but now you need to take complete advantage of them in order for them to really help you. What you need to do is run these tests, process the results, and then receive a notification if they fail. This is where we are heading in this article by Zdenek Machek, author of PHPUnit Essentials, and there are a few really good open source or free solutions available that can help you.

(For more resources related to this topic, see here.)

This article is named Continuous Integration; so, what exactly does this mean? You can find many long definitions, but to put it simply, it is a process where you integrate your code with code from other developers and run tests to verify the code functionality. You are aiming to detect problems as soon as possible and trying to fix problems immediately. It is always easier and cheaper to fix a couple of small problems than create one big problem.

This can be translated to the following workflow:

  1. The change is committed to a version control system repository (such as Git or SVN).

  2. The Continuous Integration (CI) server is either notified of, or detects a change and then runs the defined tests.

  3. CI notifies the developer if the tests fail.

With this method you immediately know who created the problem and when.

For the CI to be able to run tests after every commit point, these tests need to be fast. Usually, you can do this with unit tests for integration, and with functional tests it might be better to run them within a defined time interval, for example, once every hour.

You can have multiple sets of tests for each project, and another golden rule should be that no code is released to the production environment until all of the tests have been passed.

It may seem surprising, but these rules and processes shouldn't make your work any slower, and in fact, should allow you to work faster and be more confident about the developed code functionality and changes. Initial investment pays off when you can focus on adding new functionality and are not spending time on tracking bugs and fixing problems. Also, tested and reliable code refers to code that can be released to the production environment more frequently than traditional big releases, which require a lot of manual testing and verification. There is a real impact on business, and it's not just about the discussion as to whether it is worthwhile and a good idea to write some tests and find yourself restricted by some stupid rules anymore.

What will really help and is necessary is a CI server for executing tests and processing the results; this is also called test automation. Of course, in theory you can write a script for it and test it manually, but why would you do that when there are some really nice and proven solutions available? Save your time and energy to do something more useful.

In this article, we will see what we can do with the most popular CI servers used by the PHP community:

  • Travis CI

  • Jenkins CI

  • Xinc

For us, a CI server will always have the same main task, that is, to execute tests, but to be precise, it includes the following steps:

  1. Check the code from the repository.

  2. Execute the tests.

  3. Process the results.

  4. Send a notification when tests fail.

This is the bare minimum that a server must handle. Of course, there is much more to be offered, but these steps must be easy to configure.

Using a Travis CI hosted service

Travis is the easiest to use from the previously mentioned servers. Why is this the case? This is because you don't have to install it. It's a service that provides integration with GitHub for many programming languages, and not just for PHP. Primarily, it's a solution for open source projects, meaning your repository on GitHub is a public repository. It also has commercial support for private repositories and commercial projects.

What is really good is that you don't have to worry about server configuration; instead, you just have to specify the required configuration (in the same way you do with Composer), and Travis does everything for you. You are not just limited to unit tests, and you can even specify which database you want to use and run ingratiation tests there.

However, there is also a disadvantage to this solution. If you want to use it for a private repository, you have to pay for the service, and you are also limited with regard to the server configuration. You can specify your PHP version, but it's not recommended to specify a minor version such as 5.3.8; you should instead use a major version, such as 5.3. On the other hand, you can run tests against various PHP versions, such as PHP 5.3, 5.4, or 5.5, so when you want to upgrade your PHP version, you already have the test results and know how your code will behave with the new PHP version.

Travis has become the CI server of choice for many open source projects, and it's no real surprise because it's really good!

Setting up Travis CI

To use Travis, you will need an account on GitHub. If you haven't got one, navigate to https://github.com/ and register there. When you have a GitHub account, navigate to https://travis-ci.org/ and click on Sign in with GitHub.

As you can see in the preceding screenshot, there will be a Travis application added to your GitHub account. This application will work as a trigger that will start a build after any change is pushed onto the GitHub repository. To configure the Travis project, you have to follow these steps:

  1. You will be asked to allow Travis to access your account.

  2. When you do this you will go back to the Travis site, where you will see a list of your GitHub repositories.

  3. By clicking on On/Off, you can decide which project should be used by Travis.

  4. When you click on a project configuration, you will be taken to GitHub to enable the service hook. This is because you have to run a build after every commit, and Travis is going to be notified about this change.

  5. In the menu, search for Travis and fill in the details that you can find in your Travis account settings. Only the username and token are required, and the domain is optional.

For a demonstration, you can refer to my sample project, where there is just one test suite, and its purpose is to test how Travis works (navigate to https://github.com/machek/travis):

Using Travis CI

When you link your GitHub account to Travis and set up a project to notify Travis, you need to configure the project.

You need to follow the project setup in the same way that we did earlier. To have classes, you are required to have the test suites that you want to run, a bootstrap file, and a phpunit.xml configuration file.

You should try this configuration locally to ensure that you can run PHPUnit, execute tests, and make sure that all tests pass.

If you cloned the sample project, you will see that there is one important file: .travis.yml.

This Travis configuration file is telling Travis what the server configuration should look like, and also what will happen after each commit.

Let's have a look at what this file looks like:

# see http://about.travis-ci.org/docs/user/languages/php/ for more hints language: php # list any PHP version you want to test against php: - 5.3 - 5.4 # optionally specify a list of environments env: - DB=mysql # execute any number of scripts before the test run, custom env's are
available as variables before_script: - if [[ "$DB" == "mysql" ]]; then mysql -e "create database IF NOT EXISTS
my_db;" -uroot; fi # omitting "script:" will default to phpunit script: phpunit --configuration phpunit.xml --coverage-text # configure notifications (email, IRC, campfire etc) notifications: email: "your@email"

As you can see, the configuration is really simple, and it shows that we need PHP 5.3 and 5.4, and a MySQL database to create a database, execute the PHPUnit with our configuration, and send a report to my e-mail address.

After each commit, PHPUnit executes all the tests. The following screenshot shows us an interesting insight into how Travis executes our tests and which environment it uses:

You can view the build and the history for all builds.

Even though there are no real builds in PHP because PHP is an interpreted language and not compiled, the action performed when you clone a repository, execute PHPUnit tests, and process results is usually called a build.

Travis configuration can be much more complex, and you can run Composer to update dependency and much more. Just check the Travis documentation for PHP at http://about.travis-ci.org/docs/user/languages/php/.

Using the Jenkins CI server

Jenkins is a CI server. The difference between Travis and Jenkins is that when you use Travis as a service, you don't have to worry about the configuration, whereas Jenkins is piece of software that you install on your hardware. This is both an advantage and a disadvantage. The disadvantage is that you have to manually install it, configure it, and also keep it up to date. The advantage is that you can configure it in a way that suits you, and all of the data and code is completely under your control. This can be very important when you have customer code and data (for testing, never use live customer data) or sensitive information that can't be passed on to a third party.

The Jenkins project started as a fork of the Hudson project and is written in Java but has many plugins that suit a variety of programming languages, including PHP. In recent years, it has become very popular, and nowadays it is probably the most popular CI server. The reasons for its popularity are that it is really good, can be configured easily, and there are many plugins available that probably cover everything you might need.

Installation

Installation is a really straightforward process. The easiest method is to use a Jenkins installation package from http://jenkins-ci.org/. There are packages available for Windows, OS X, and Linux, and the installation process is well-documented there. Jenkins is written in Java, which means that Java or OpenJDK is required. After this comes the installation, as you just launch the installation and point it to where it should be installed, and Jenkins is listening on port 8080.

Before we move on to configure the first project (or job in Jenkins terminology), we need to install a few extra plugins. This is Jenkins' biggest advantage. There are many plugins and they are very easy to install. It doesn't matter that Jenkins is a Java app as it also serves PHP very well.

For our task to execute tests, process results, and send notifications, we need the following plugins:

  • Email-ext: This plugin is used to send notifications

  • Git or Subversion: This plugin is used to check the code

  • xUnit: This plugin is used for processing the PHPUnit test results

  • Clover PHP: This plugin is used for processing the code coverage

To install these plugins, navigate to Jenkins | Manage Jenkins | Manage Plugins and select the Available tab. You can find and check the required plugins, or alternatively use the search filter to find the one you need:

For e-mails, you might need to configure the STMP server connection at Manage Jenkins | Configure System | E-mail notification section.

Usage

By now, we should have installed everything that we need, and we can start to configure our first simple project. We can use the same simple project that we used for Travis. This is just one test case, but it is important to learn how to set up a project. It doesn't matter if you have one or thousands of tests though, as the setup is going to be the same.

Creating a job

The first step is to create a new job. Select New Job from the Jenkins main navigation window, give it a name, and select Build a free-style software project. After clicking on OK, you get to the project configuration page.

The most interesting things there are listed as follows:

  • Source Code Management: This is where you check the code

  • Build Triggers: This specifies when to run the build

  • Build: This tests the execution for us

  • Post-build Actions: This publishes results and sends notifications

The following screenshot shows the project configuration window in Jenkins CI:

Source Code Management

Source code management simply refers to your version control system, path to the repository, and the branch/branches to be used. Every build is a clean operation, which means that Jenkins starts with a new directory where the code is checked.

Build Triggers

Build triggers is an interesting feature. You don't have to use it and you can start to build manually, but it is better to specify when a build should run. It can run periodically at a given interval (every two hours), or you can trigger a build remotely.

One way to trigger a build is to use post commit hooks in the Git/SVN repository. A post commit hook is a script that is executed after every commit. Hooks are stored in the repository in the /hooks directory (.git/hooks for Git and /hooks for SVN). What you need to do is create a post-commit (SVN) or post-receive (Git) script that will call the URL given by Jenkins when you click on a Trigger remotely checkbox with a secret token:

#!/bin/sh wget
http ://localhost:8080/job/Sample_Project/build?token=secret12345ABC-O /dev/null

After every commit/push to the repository, Jenkins will receive a request to run the build and execute the tests to check whether all of the tests work and that any code change there is not causing unexpected problems.

Build

A build is something that might sound weird in the PHP world, as PHP is interpreted and not compiled; so, why do we call it a build? It's just a word. For us, it refers to a main part of the process—to execute unit tests.

You have to navigate to Add a build step—click on either Execute Windows batch command or Execute shell. This depends on your operating system, but the command remains the same:

phpunit --log-junit=result.xml --coverage-clover=clover.xml

This is simple and outputs what we want. It executes tests, stores the results in the JUnit format in the file result.xml, and generates code coverage in the clover format in the file clover.xml.

I should probably mention that PHPUnit is not installed with Jenkins, and your build machine on which Jenkins is running must have PHPUnit installed and configured, including PHP CLI.

Post-build Actions

In our case, there are three post-build actions required. They are listed as follows:

  • Process the test result: This denotes whether the build succeeded or failed. You need to navigate to Add a post-build action | Publish Junit test result report and type result.xml. This matches the switch --log-junit=result.xml. Jenkins will use this file to check the tests results and publish them.

  • Generate code coverage: This is similar to the first step. You have to add the Publish Clover PHP Coverage report field and type clover.xml. It uses a second switch, --coverage-clover=clover.xml, to generate code coverage, and Jenkins uses this file to create a code coverage report.

  • E-mail notification: It is a good idea to send an e-mail when a build fails in order to inform everybody that there is a problem, and maybe even let them know who caused this problem and what the last commit was. This step can be added simply by choosing E-mail notification action.

Results

The result could be just an e-mail notification, which is handy, but Jenkins also has a very nice dashboard that displays the current status for each job, and you can also see and view the build history to see when and why a build failed.

A nice feature is that you can drill down through the test results or code coverage and find more details about test cases and code coverage per class.

To make testing even more interesting, you can use Jenkins' The Continuous Integration Game plugin. Every developer receives positive points for written tests and a successful build, and negative points for every build that they broke. The game leaderboard shows who is winning the build game and writing better code.

PHPUnit Essentials Get started with PHPUnit and learn how to write and test codes using advanced technologies with this book and ebook
Published: May 2014
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

Using the Xinc PHP CI server

Xinc is a CI server written in PHP. What is interesting about this server is that it's in PHP, which means it's going to be a lightweight solution compared to Java applications, and when you have to pay for cloud service, then the amount of memory matters. With Xinc, 512 MB of memory might be enough, whereas for Java you will need at least 1 GB of memory. Another advantage is that it's PHP, so if there is any problem or you want to tweak or extend it, then it's easy.

A disadvantage is that this system is not used as widely as the previous two solutions, and its configuration might also look a bit confusing. Yet, it has pretty good documentation, and so it shouldn't be too difficult to overcome these problems.

Xinc has two parts. The first part is Xinc running as a background process and checking repositories for changes and starting builds, while the second part is a web application that provides an interface to the test results.

Installation

The recommended installation method is to use a PEAR installer. There are a few required dependencies, so manual installation might be a bit tricky.

For installation, you need to discover a few PEAR channels and then run the installation:

>pear config-set auto_discover 1 >pear channel-discover pear.phpunit.de >pear channel-discover pear.elektrischeslicht.de >pear channel-discover components.ez.no >pear install --alldeps xinc/Xinc

After installation, you need to run the following post installation script:

>pear run-scripts xinc/Xinc

This script gives you a few options about where and how Xinc will be installed. You can press the Enter key to use the default installation:

After installation, you will get instructions to enable Xinc to operate on your machine, and they will look similar to this:

Xinc installation complete. - Please include /etc/xinc/www.conf in your apache virtual hosts. - Please enable mod-rewrite. - To add projects to Xinc, copy the project xml to /etc/xinc/conf.d - To start xinc execute: sudo /etc/init.d/xinc start UNINSTALL instructions: - pear uninstall xinc/Xinc - run: /usr/bin/xinc-uninstall to cleanup installed files [OK] Saved ini settings Install scripts complete

After installation, it is a good idea to check whether Xinc actually works because if there is a problem, it might be really tricky to debug what's going on. You can check it by running the following line of code:

>/usr/bin/xinc

One of the common problems that you can see now is this error message:

PHP Warning: require(Xinc.php): failed to open stream: No such file or
directory in /usr/bin/xinc on line 32
PHP Stack trace: PHP 1. {main}() /usr/bin/xinc:0 PHP Fatal error: require(): Failed opening required 'Xinc.php'
(include_path='.:/root/pear/share/pear') in /usr/bin/xinc on line 32
PHP Stack trace: PHP 1. {main}() /usr/bin/xinc:0

This problem has a simple solution; check whether in your php.ini file, PEAR is added to include_path.

After these steps, you will have Xinc running as a background task and listening on port 8080 as a web application (you can change it in www.conf if you don't like it), and you will have also installed a sample project.

Usage

After installation, we will focus on the aforementioned Simple Project. The configuration file is located at /etc/xinc/conf.d/simpleproject.xml. Xinc uses Phing (for more information, refer to http://www.phing.info). This is a PHP clone of Ant—a building tool that drives processes. You can think of it as a script that says run this and then this. It might sound a bit strange to learn that it uses XML, but when you get used to it, you will realize it's quite an interesting way to go about things.

Ant is far more popular than Phing, but the logic and structure is the same or very similar, and Phing gives you an advantage as there is an option to extend it with PHP if required.

Let's take this simple project and modify it for the example that we used earlier by following the same steps:

  1. Check the Git repository.

  2. Run the PHPUnit tests.

  3. Publish the results.

  4. Send notifications.

Almost everything is in place for a sample project, and there are two important files at the location /etc/xinc/conf.d/sampleproject.xml:

<?xml version="1.0"?> <xinc> <project name="SimpleProject"> <configuration> <setting name="loglevel" value="1"/> <setting name="timezone" value="US/Eastern"/> </configuration> <property name="dir" value="/var/xinc/projects/SimpleProject/"/> <cron timer="*/4 * * * *"/> <modificationset> <buildalways/> </modificationset> <builders> <phingBuilder buildfile="${dir}/build.xml" target="build"/> </builders> <publishers> <phpUnitTestResults file="${dir}/report/logfile.xml"/> <onfailure> <email to="root" subject="${project.name} build ${build.number}
failed"message="The build failed."/> </onfailure> <onsuccess> <phingPublisher buildfile="${dir}/publish.xml" target="build"/> <artifactspublisher file="${dir}/publish.xml"/> <artifactspublisher file="${dir}/publish.xml"/> <deliverable file="${dir}/builds/release-${build.label}.tar.gz"
alias="release.tar.gz"/> </onsuccess> <onrecovery> <email to="root" subject="${project.name} build ${build.number}
was recovered"message="The build passed after having failed before."/> </onrecovery> </publishers> </project> </xinc>

This is a project configuration file that checks for modifications every 4 minutes. Run the build, publish the results, and send notifications when the build fails or recovers. The main configuration is in the following line:

<phingBuilder buildfile="${dir}/build.xml" target="build"/>

The preceding line says the build configuration file is sitting inside the project directory and is called build.xml. Let's have a look at what's there:

<?xml version="1.0"?> <project name="Simple Project Build File"
basedir="/var/xinc/projects/SimpleProject" default="build"> <property name="report.dir" value="${project.basedir}/report"/> <target name="build" depends="prepare, test, tar, generate-report"> </target> <target name="prepare"> <mkdir dir="${report.dir}"/> </target> <target name="tar"> <tar destfile="${project.basedir}/release-${xinc.buildlabel}.tar.gz"
compression="gzip"> <fileset dir="."> <include name="index.php" /> <include name="Page.php" /> </fileset> </tar> </target> <target name="test"> <phpunit haltonfailure="true" printsummary="true"> <batchtest> <fileset dir="."> <include name="*Test.php"/> </fileset> </batchtest> <formatter type="xml" todir="${report.dir}"
outfile="logfile.xml"/> </phpunit> </target> <target name="generate-report"> <phpunitreport infile="${report.dir}/logfile.xml"
styledir="resources/xsl" todir="report" format="noframes"/> </target> </project>

This is actually a Phing script. In Phing (Ant) terminology, tasks are called targets, and the dependency specifies what tasks need to be executed. For this build, the tasks that are going to be executed are prepare, test, tar, and generate-report targets.

This is what we need with one exception; we need to check the Git repository. The first step is manual, but it could be part of the build. We create/code the directory and clone the GitHub repository we used for previous tests:

git clone https://github.com/machek/travis

To be able to use Git with Phing, we have to install the Git plugin:

>pear install VersionControl_Git-alpha

Now, we need to slightly modify the build script to pull from the Git repository and execute our tests:

<?xml version="1.0"?> <project name="Simple Project Build File"
basedir="/var/xinc/projects/SimpleProject" default="build"> <property name="report.dir" value="${project.basedir}/report"/> <property name="code.dir" value="${project.basedir}/code"/> <target name="build" depends="prepare, gitpull, test, tar,
generate-report"> </target> <target name="prepare"> <mkdir dir="${report.dir}"/> <mkdir dir="${code.dir}"/> </target> <target name="gitpull"> <echo msg="Getting latest code from ${git.repo}" /> <gitpull gitPath="git" repository="${code.dir}" all="true" /> </target> <target name="tar"> <tar destfile="${project.basedir}/release-${xinc.buildlabel}.tar.gz"
compression="gzip"> <fileset dir="."> <include name="index.php" /> <include name="Page.php" /> </fileset> </tar> </target> <target name="test"> <phpunit haltonfailure="true" printsummary="true"
configuration="${code.dir}/phpunit.xml"> <batchtest> <fileset dir="${code.dir}/tests"> <include name="*Test.php"/> </fileset> </batchtest> <formatter type="xml" todir="${report.dir}"
outfile="logfile.xml"/> </phpunit> </target> <target name="generate-report"> <phpunitreport infile="${report.dir}/logfile.xml"
styledir="resources/xsl" todir="report" format="noframes"/> </target> </project>

There is one extra target, gitpull, now, and the target tests were also changed to use our tests. This is all that you need to set up a project. When you access the Xinc dashboard served by Apache, you should see a result similar to the following:

Summary

A continuous integration server is something you really need. You can run PHPUnit tests from a command line, but it's much nicer when it happens automatically, as you have a history of test results and code coverage, and you can notify developers when somebody breaks the tests. It's like a cherry on top, but there is something satisfying when you see that all of your hard work makes sense and you can see its results. Drill through the test results and see the code coverage for each class and method.

We saw three examples, which are probably the most interesting solutions in the PHP world. Travis is very easy to configure and use as a service for open source projects. The king of CI servers is Jenkins, with tons of plugins and easy configuration, and the Xinc CI server is written in PHP. It might depend on your project, but you should try to choose the right one for you. Of course, you can do much more than just run unit tests. You might need a Selenium Server, and then the question is whether your CI server will also host a Selenium Server. You can try to find out more about your code with tools such as PHP CodeSniffer in order to detect if your developers are following coding standards. There is so much more that you can do with CI servers, so just give it a try. You will love them!

Resources for Article:


Further resources on this subject:


PHPUnit Essentials Get started with PHPUnit and learn how to write and test codes using advanced technologies with this book and ebook
Published: May 2014
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

About the Author :


Zdenek Machek

Zdenek Machek is an experienced PHP developer, who has been working with PHP since the year 2000 and PHP3 days. He introduced software testing and PHPUnit to various companies, and used them on various small as well as large scale projects.

Zdenek wrote several articles and blog posts focused on Continuous Integration processes during PHP application development.

Currently, Zdenek leads technology standards and values across the organization, and also handles analysis, planning, and technical delivery of large scale, critical, and high performance systems for our most complex projects.

Books From Packt


 Learning FuelPHP for Effective PHP Development
Learning FuelPHP for Effective PHP Development

PHP Web 2.0 Mashup Projects: Practical PHP Mashups with Google Maps, Flickr, Amazon, YouTube, MSN Search, Yahoo!
PHP Web 2.0 Mashup Projects: Practical PHP Mashups with Google Maps, Flickr, Amazon, YouTube, MSN Search, Yahoo!

Building Websites with PHP-Nuke
Building Websites with PHP-Nuke

PHP Team Development
PHP Team Development

 Instant PhpStorm Starter [Instant]
Instant PhpStorm Starter [Instant]

Expert PHP 5 Tools
Expert PHP 5 Tools

CakePHP Application Development
CakePHP Application Development

phpMyAdmin Starter [Instant]
phpMyAdmin Starter [Instant]


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software