Functional Testing with JMeter

Exclusive offer: get 50% off this eBook here
Apache JMeter

Apache JMeter — Save 50%

A practical beginner's guide to automated testing and performance measurement for your websites

£14.99    £7.50
by Emily H. Halili | August 2008 | Java Open Source

JMeter is a 100% pure Java desktop application. JMeter is found to be very useful and convenient in support of functional testing. Although JMeter is known more as a performance testing tool, functional testing elements can be integrated within the Test Plan, which was originally designed to support load testing. Many other load-testing tools provide little or none of this feature, restricting themselves to performance-testing purposes. Besides integrating functional-testing elements along with load-testing elements in the Test Plan, you can also create a Test Plan that runs these exclusively. In other words, aside from creating a Load Test Plan, JMeter also allows you to create a Functional Test Plan. This flexibility is certainly resource-efficient for the testing project.

In this article by Emily H. Halili, we will give you a walkthrough on how to create a Test Plan as we incorporate and/or configure JMeter elements to support functional testing.

Preparing for Functional Testing

JMeter does not have a built-in browser, unlike many functional-test tools. It tests on the protocol layer, not the client layer (i.e. JavaScripts, applets, and many more.) and it does not render the page for viewing. Although, by default that embedded resources can be downloaded, rendering these in the Listener | View Results Tree may not yield a 100% browser-like rendering. In fact, it may not be able to render large HTML files at all. This makes it difficult to test the GUI of an application under testing.

However, to compensate for these shortcomings, JMeter allows the tester to create assertions based on the tags and text of the page as the HTML file is received by the client. With some knowledge of HTML tags, you can test and verify any elements as you would expect them in the browser.

It is unnecessary to select a specific workload time to perform a functional test. In fact, the application you want to test may even reside locally, with your own machine acting as the "localhost" server for your web application. For this article, we will limit ourselves to selected functional aspects of the page that we seek to verify or assert.

Using JMeter Components

We will create a Test Plan in order to demonstrate how we can configure the Test Plan to include functional testing capabilities. The modified Test Plan will include these scenarios:

    1. Create Account—New Visitor creating an Account
    2. Login User—User logging in to an Account

Following these scenarios, we will simulate various entries and form submission as a request to a page is made, while checking the correct page response to these user entries. We will add assertions to the samples following these scenarios to verify the 'correctness' of a requested page. In this manner, we can see if the pages responded correctly to invalid data. For example, we would like to check that the page responded with the correct warning message when a user enters an invalid password, or whether a request returns the correct page.

First of all, we will create a series of test cases following the various user actions in each scenario. The test cases may be designed as follows:

CREATE ACCOUNT

Test Steps

Data

Expected

1

Go to Home page.

www.packtpub.com

Home page loads and renders with no page error

2

Click Your Account link (top right).

User action

1. Your Account page loads and renders with no page error.2. Logout link is not found.

3

No Password: - Enter email address in Email text field.- Click the Create Account and

Continue button.

email=EMAIL

1. Your Account page resets with Warning message-Please enter password.2. Logout link not found.

4

Short Password: - Enter email address in Email text field.- Enter password in Password text field.- Enter password in

Confirm Password text field.

- Click Create Account and

Continue button.

email=EMAILpassword=SHORT_PWD

confirm password=SHORT_PWD

1. Your Account page resets with Warning message-Your password must be 8 characters or longer.2. Logout link is not found.

5

Unconfirmed Password: - Enter email address in Email text field.- Enter password in Password text field.- Enter password in

Confirm Password text field.

- Click Create Account and

Continue button.

email=EMAILpassword=VALID_PWDconfirm password=INVALID_PWD

1. Your Account page resets with Warning messagePassword does not match.2. Logout link is not found.

6

Register Valid User: - Enter email address in Email text field.- Enter password in Password text field.- Enter password in

Confirm Password text field.

- Click Create Account and

Continue button.

email=EMAILpassword=VALID_PWDconfirm password=VALID_PWD

1. Logout link is found.2. Page redirects to User Account page.3. Message found: You are registered as: e:<EMAIL>.

7

Click Logout link.

User action

1. Logout link is NOT found.

 

LOGIN USER

Test Steps

Data

Expected

1

Click Home page.

User action

1. WELCOME tab is active.

2

Log in Wrong Password: - Enter email in Email text field- Enter password at Password text field.- Click Login button.

email=EMAILpassword=INVALID_PWD

1. Logout link is NOT found.2. Page refreshes.3. Warning message-Sorry your password was incorrect appears.

3

Log in Non-Exist Account:- Enter email in Email text field.- Enter password in Password text field.- Click Login button.

email=INVALID_EMAILpassword=INVALID_PWD

1. Logout link is NOT found.2. Page refreshes.3. Warning message-Sorry, this does not match any existing accounts. Please check your details and try again or open a new account below appears.

4

Log in Valid Account:- Enter email in Email text field.- Enter password in Password text field.- Click Login-button.

email=EMAILpassword=VALID_PWD

1. Logout link is found.2. Page reloads.3. Login successful message-You are logged in as: appears.

5

Click Logout link.

User action

1. Logout link is NOT found.


Apache JMeter A practical beginner's guide to automated testing and performance measurement for your websites
Published: June 2008
eBook Price: £14.99
Book Price: £24.99
See more
Select your format and quantity:

With the exception of the Configuration elements, Listeners, and Assertions,which we will add later, our Test Plan will take the form that you see in thefollowing screenshot:

Functional Testing with JMeter

Using HTTP Proxy Server to Record Page Requests

You will need to include the HTTP Proxy Server element in the WorkBench. Some configuration will be required, as shown in the following snapshot:

Functional Testing with JMeter

Configuring the Proxy Server

Simulating Create Account and Login User scenarios will require JMeter to make requests for the registration and login pages that are exposed via HTTPS. By default, HTTP Proxy Server is unable to record HTTP requests. However, we can override this by selecting (checking) the Attempt HTTPS Spoofing checkbox.

Selecting Add Assertion will be especially useful as we add specific patterns of the page that we want to evaluate as a later part of this exercise. The Capture HTTP Headers option is selected to capture the Header information as we begin recording. However, to make the recording neater, we will keep this option unchecked.

In addition, since we do not require images in our testing, in the URL Pattern to Exclude section, add these patterns: .*.jpg, .*.js, .*.png, .*.gif', .*.ico, .*.css, otherwise these image files, which are not necessary for our testing, will be recorded causing unnecessary clutter in our recording.

 

 

Adding HTTP Request Default

A useful addition to this element is the HTTP Request Default element, a type of Configuration element. Since this Test Plan will employ multiple HTTP request elements targeting the same server and port, this element will be very useful. The web server name will not be captured for each HTTP Request sampler record, since the Request Default element will retain this information. With a little configuration change in this element, it allows the Test Plan to run even when the application is the deployed to a different server and/or port. The following snapshot is the HTTP Request Default element that we will use for this exercise.

Functional Testing with JMeter

As we use this default element, our subsequent recording never needs to appendthe Server name. The result of our recording of the first page is shown in the following snapshot:

Functional Testing with JMeter

Adding HTTP Header Manager

Another very useful default element is the HTTP Header Manager Configuration element. This element can either be added to the Test Plan and configured manually as an afterthought, or we can simply use the recorded Browser-derived headers element as included in the recording. For convenience, we will choose the latter option. Once the Proxy Server records the homepage request, stop the recording. You will find a Header Manager for this page is being captured, as Browser-derived header. Simply click and drag this element to the top of the current scope of the HTTP Proxy Server. Notice that I have removed the Referer, since we want to create a default for the remaining HTTP Requests. Following is a snapshot of this change.

Functional Testing with JMeter

Now,you may de-select the Capture HTTP Headers option in the Proxy Server element, since we have the default header.

Let the Recording Begin...

Let us proceed with the recording following the test cases in the previous table as our guide. As you record each page, select the specific tags or page elements the correctness of which you want to validate and add them to the Patterns to Test section in the Response Assertion element of each sampler. This may take most of your recording time, since as you record, you need to decide carefully which page element(s) would be the most effective measure of correctness. There are plenty of developer tools available to help you in this possibly tedious task. My favorite is the Inspect Element feature in Firebug, a Firefox browser add-on by Mozilla. You may choose patterns that you would expect to see or otherwise by selecting or de-selecting the Not option at Pattern Matching Rules section.

After recording is completed, you may rename and organize your samplers, as you move them to the Test Plan (refer to the following figure). You may want to add a few more Configuration elements in your Test Plan, as in my sample shown in the following snapshot:

Functional Testing with JMeter

  • User Defined Variables have been added, two more Listeners, and a Constant Timer with a constant delay of 2 seconds after the request for each page was completed. The Assertion Results listener is used with the Response Assertion elements, to summarize the success or failure of a page in meeting the validation criteria defined in each Response Assertion.

Adding User Defined Variables

The User Defined Variables (UDV) element as shown in the following snapshot is particularly interesting with regards to the test case design we drafted earlier in the table. It allows you to plug values to variables being used in various locations in the Test Plan. The JMeter Test Plan we have created will implement the exact values assigned to different variables. Following is a snapshot of the UDV I have set up for our Test Plan.

Functional Testing with JMeter

How do we use these variables in the Test Plan? Simply use the format ${Variable-name} anywhere in the Test Plan that we want to use the value of a Variable.

For example, in the HTTP Request Sampler following CREATE ACCOUNT | Test Step#6: Register Valid User, as you can see below, the parameter password has value ${VALID_PWD}, referring to the corresponding variable assigned in UDV.

Functional Testing with JMeter

We may also use the variables set in UDV in other elements, namely Response Assertions. This feature is particularly useful when the assertion depends on varying values, such as when we want to verify URLs, verifying user names, account no, etc.—depending on the values we want to include throughout the entire testing. The following snapshot may give us a clear idea of how a UDV can be used in an Assertion element. The URL variable defined in UDV is used in the Patterns to Test section of this Assertion, as part of a complete page element that we want to verify in the page Sampler.

Functional Testing with JMeter

Running the Test

Once the assertions are properly completed, we are expecting that running our Test Plan would pass all the assertions. Passed assertions will not show any error in Assertion Results | Listener installed within the same scope. As for all Listeners, results as captured by the Listeners can be saved and reproduced at a later time. Following is a sample explaining what passed Assertions would reveal as the Testis executed.

Functional Testing with JMeter

On the other hand, a failed Assertion would show an error message in the same Listener as the following snapshot illustrates.

Functional Testing with JMeter

Since a page error or Page not found error is a real risk in web applications, a failure may originate from such an error, and not just because of a failed Assertion. We can view more information about the sampler that contains the failed Assertion to investigate the origins of a failure. A View Results Tree Listener records the details of requests and logs all errors (indicated by the red warning sign and red fonts).

The following figure shows that the page was available and page request was successful, however, the assertion failed.

Functional Testing with JMeter

Summary

This article provided visual means for you to understand the capabilities of JMeter tools that support functional testing, as we directly wrote and implemented a JMeter script. We have demonstrated building a Test Plan to contain functional validations (or assertions) by incorporating various essential JMeter components, particularly the 'Response Assertion' element and 'Assertion Result' Listener. By using the 'User Defined Variable' Configuration element, we have also parameterized several values in order to give our Test Plan better flexibility. In addition, we have observed the result of these assertions as we performed a 'live' run of the application under test. An HTTP Request sampler may require to be modified, if there are any changes to the parameter(s) that the sampler sends with each request. Once created, a JMeter Test Plan that contains assertions can then be used and modified in subsequent Regression tests for the application.



Apache JMeter A practical beginner's guide to automated testing and performance measurement for your websites
Published: June 2008
eBook Price: £14.99
Book Price: £24.99
See more
Select your format and quantity:

About the Author :


Emily H. Halili

Since graduating in 1998, from California State University in Computer Science, Emily H. Halili has taken numerous roles in the IT/Software industry—namely as Software Engineer, Network Engineer, Lecturer, and Trainer. Currently a QA Engineer in CEO Consultancy-Malaysia with great passion for testing, she has two years of experience in software testing and managing QA activities. She is an experienced manual tester and has practical knowledge of various open-source automation tools and framework, including JMeter, Selenium, JProfiler, Badboy, Sahi, Watij and many more.

Books From Packt

Apache JMeter
Apache JMeter

Swing Extreme Testing
Swing Extreme Testing

EJB 3 Developer Guide
EJB 3 Developer Guide

Object-Oriented JavaScript
Object-Oriented JavaScript

ZK Developer’s Guide
ZK Developer’s Guide

Alfresco Enterprise Content Management Implementation
Alfresco Enterprise Content Management Implementation

OpenCms 7 Development
OpenCms 7 Development

Service Oriented Architecture with Java
Service Oriented Architecture with Java


 

Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software