Home Web Development UI Testing with Puppeteer

UI Testing with Puppeteer

By Dario Kondratiuk
books-svg-icon Book
eBook $26.99 $17.99
Print $38.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $26.99 $17.99
Print $38.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Chapter 2: Automated Testing and Test runners
About this book
Puppeteer is an open source web automation library created by Google to perform tasks such as end-to-end testing, performance monitoring, and task automation with ease. Using real-world use cases, this book will take you on a pragmatic journey, helping you to learn Puppeteer and implement best practices to take your automation code to the next level! Starting with an introduction to headless browsers, this book will take you through the foundations of browser automation, showing you how far you can get using Puppeteer to automate Google Chrome and Mozilla Firefox. You’ll then learn the basics of end-to-end testing and understand how to create reliable tests. You’ll also get to grips with finding elements using CSS selectors and XPath expressions. As you progress through the chapters, the focus shifts to more advanced browser automation topics such as executing JavaScript code inside the browser. You’ll learn various use cases of Puppeteer, such as mobile devices or network speed testing, gauging your site’s performance, and using Puppeteer as a web scraping tool. By the end of this UI testing book, you’ll have learned how to make the most of Puppeteer’s API and be able to apply it in your real-world projects.
Publication date:
March 2021
Publisher
Packt
Pages
316
ISBN
9781800206786

 

Chapter 2: Automated Testing and Test runners

In Chapter 1, Getting started with Puppeteer, we covered the first fundamental pillar of this book: browser automation and headless browsers. In this chapter, we are going to cover the second pillar: UI testing. We learned that Puppeteer is not just about testing, but that doesn't mean that it's not an excellent tool for the job.

In this chapter, we are going to learn the fundamentals of Testing Automation. We are going to see the differences between UI Testing and End-to-End testing. If you have tried to write tests in Node.js before, you might have come across some weird names: Mocha, Jest, Jasmine, AVA, or Chai. That feels quite overwhelming if you are not used to these tools. We are going to see which are the right tools for us.

We will cover the following topics in this chapter:

  • Introduction to Automated Testing
  • Test runner main features
  • Available Test runners
  • Creating our first test project
  • Organizing our code

Once we understand these foundational concepts and we learn how test runners work, we will be able to dive deep into the Puppeteer API.

 

Technical requirements

You will find all the code of this chapter on the GitHub repository (https://github.com/PacktPublishing/UI-Testing-with-Puppeteer) under the Chapter2 directory.

 

Introduction to Automated Testing

Testing is a fundamental task in software development. Even if you consider yourself a bad tester, or even a bad developer, you do some testing when you code your app. At the very least, you open the app to see whether it works as expected.

Maybe you are a little bit more methodical and you have a test plan, at least in your mind. You know that when you code a form, you have to validate some common scenarios:

  • Try to save a form with empty fields.
  • Try to save with good data.
  • Try to enter bad data. You might enter text in numeric fields, invalid dates, and so on.

More experienced developers will cover all the possible scenarios. They will write code based on those scenarios and then test accordingly.

Then we get to the word that's driving this book: we automate stuff. We want to automate our tests. We don't want to forget any scenarios or have to test the same thing over and over.

As you will notice, I haven't mentioned Quality Assurance (QA) analysts yet, because I want to highlight that testing is not something relegated to the QA team. Those who are involved in the testing process include the following:

  • Backend developers
  • Frontend developers
  • QA analysts
  • Managers (product or project managers)

We need to know that there are different types of tests. Some types of tests will be performed by developers and QA analysts. Other tests will be specific to either developers or QA analysts.

Mike Cohn, in his book Succeeding with Agile (Addison-Wesley Professional), introduced his very popular Testing Pyramid:

Mike Cohn’s Testing Pyramid

Mike Cohn's Testing Pyramid

Although Mike's book is more than 10 years old, this pyramid is still valid.

This pyramid is based on three characteristics:

  • Number of tests
  • Isolation
  • Speed

I have only one thing against this pyramid: the word UI. Modern apps rely more and more on client code, "UI" code. Frameworks such as React, Angular, and Vue.js allow developers to write reusable components. Many apps now have most of their business rules running on the client.

Frontend developers should not be limited to the top of this pyramid. They should be able to write unit tests and service tests for their UI code. This might look like a small change, but I think it's important. With this change in the paradigm, we get a pyramid that looks like this:

New Pyramid

New Pyramid

Now that we have a better understanding, let's talk about the different levels of this pyramid.

Unit tests

Unit tests are the base of the pyramid. The more business logic you cover in unit tests, the less ground you will need to cover in service or UI tests.

As we can see in the pyramid, unit tests need to be fast and isolated. That means that a good unit test shouldn't depend on the environment or any other function. Sometimes this is easier said than done. For instance, if you want to test that the total amount of an invoice is equal to the sum of its items, you should be able to test that specific functionality in the code, without launching a web server or getting data from a database.

What roles use Unit tests?

Backend developers: For sure, Unit tests are for them. They follow the Test-Driven Development (TDD) process if possible. TDD is a technique in software development where tests are written even before any source code has been written. Once the tests have been written, the developer will program the source code to make them pass.

Frontend developers: Writing unit tests was almost impossible in the past. If you didn't have the right tools, you couldn't do your job correctly. But now, many modern libraries support unit testing. If you use React and Redux, you will find that Redux has a way to write unit tests for your components (https://www.hardkoded.com/ui-testing-with-puppeteer/redux-unit-tests).

That's not all. In the same way that backend developers need to think about how to make their code testable, if frontend developers, using modern frameworks, start creating small and testable components, they should be able to use Puppeteer to write UI unit tests. And here is where the "UI" at the top of the testing pyramid stops making any sense. Now we can write UI unit tests.

We can run a small test, rendering a component and testing, for instance, that it "renders a textbox and when I enter a value, the label below changes," or "if I pass a list of 10 items, 10 elements are rendered."

We moved UI testing to the bottom of the testing pyramid.

QA Analysts are not involved yet. Unit tests are about testing the internal code.

How about Managers? If you are a developer, I believe you are going to show this paragraph to your boss. Managers won't write unit tests, but they need to know the importance of writing unit tests and investing time in them.

These are the four benefits you (or your boss) need to know about.

Unit tests show how the code works

Unit tests explain how the code works. When I review code, I start by reviewing unit tests. If I find unit tests saying, for instance, "Create order should send email". I could read that test first, and then, check how that rule was implemented.

Business Analysts or project managers could read these tests and see whether there are any scenarios that haven't been covered or some missing validation.

Unit tests make refactoring possible

I took a risk using the word possible. But I believe that's true. You can't refactor your code if you don't have unit tests backing your changes. Remember, refactoring is changing the implementation of your code without changing the result given specific inputs. Unit tests guarantee that premise.

Unit tests prevent regressions

Regression is an involuntary change in the expected behavior of an app. If we have a good set of tests, they will prevent us from breaking any behavior of the app while we implement new features or while fixing bugs.

How can I make sure that some other developers won't come and break the precious function I just wrote? By writing unit tests. A unit test is a version of you in the future enforcing how a piece of code should work. "Create an order should send an email" – no one will be able to break that rule.

When I review code, changes in unit tests are a red flag to me. I'm not saying that unit tests shouldn't change. But if a test changes, there must be an explanation. Now, the "Create and order should send an email" shows that the sent email count is 2. Is that right? Are we sending another email? Or do we have a regression? Pay attention to changes in unit tests.

Time to go up in the test pyramid.

Service tests

Service tests are also known as Integration Tests. These tests will check how your code interacts with other components. When we talk about components, we are talking about the following:

  • Databases
  • Other components in the app
  • External services

Frontend developers would also need to integrate their code with the following:

  • Other UI components
  • CSS files
  • REST APIs

As we mentioned before, when we go up in the testing pyramid, tests become slower and less stable. And it's supposed to be like that. You will be connecting to a real database or interacting with a real REST API that would use real network calls. That would also mean that your tests would expect the environment to respond in a certain way. For instance, you would expect the database to have some set of data ready to be used, or a REST API to be available.

That's why the more tests you have in the unit test layer, the fewer integration tests you will need to code.

Let's take, for instance, the class that sends an email, could you code an integration test for that? Sure. You set up a local email server that would write emails in a temp folder, so after creating an order, you could check that folder and see whether the email server processed the email your app should have sent. But, as you can see, these kinds of orchestrations are harder to code than small unit tests.

Why do we need integration tests? Why don't we code unit tests only?

Well, you need to tests your integrations. Your code won't run in isolation. If you are testing the backend, you need to see how the database reacts to the data you are inserting, or whether a SQL query returns the data you expect.

If you are a frontend developer, this is where you would invest most of your time, checking how your component interacts on a page or how the HTML being generated affects other elements in the DOM. You would need to test how your component is being rendered with a real REST endpoint, instead of using a dummy JSON file.

What roles use Integration tests?

Backend developers: I've heard people say that these are the only tests that matter. Although I disagree with that strong opinion, I do believe these tests are essential. Say I created a unit test where, for instance, when I call CreateOrder, I get a new Order object. But now, I need to test that when I make a POST request to /orders, an order is created in the database.

Frontend developers will create tests to check how all the different components interact with each other on a page. Again, it's UI testing down in the testing pyramid.

QA Analysts will create tests similar to the tests backend and frontend developers create but with a different perspective.

Developers and QA Analysts create the same kinds of tests but with a different perspective.

Developers will create tests to back their job, so they can check whether they broke anything. And, as we mentioned before, they need tests to be able to refactor their code in the future.

QA Analysts will create tests to guarantee the application quality to the stakeholders.

There is one interesting type of test that QA Analysts can implement in this layer: the Visual Regression Test. These tests are used when we want to check whether there was any visual change regarding the style of the app. We don't want to check whether there is a button, or whether that button works. We want to check whether the button looks like how it was before. How can we achieve that? By comparing images. This technique is based on four steps:

  1. We take a screenshot as a baseline:
    Baseline image

    Baseline image

  2. We make a change in the code.
  3. We take another screenshot:
    Image after making a change

    Image after making a change

  4. We compare both images:
Differences

Differences

This type of test can be quite unstable. I bet you have seen that pages sometimes "move" when they are loading, so you have to be very sure when the page is ready for a screenshot. But it is doable. Another downside is that for every error you get, you have to analyze whether the change was a regression (a change made by mistake) or we are in the presence of a new baseline.

The role of managers is still important. They need to provide the tools and the time for developers to implement the required integration tests. They will also help QA Analysts to determine what the integrations to test are.

And so we come to the top of the pyramid, the end-to-end tests.

End-to-end tests

You might also find these tests referred to as E2E tests. The goal of E2E tests is to guarantee that an application works as expected through the entire workflow. Most applications will have more than one workflow. That would mean that it will require a number of E2E tests to cover all the possible workflows or scenarios.

Let's take a cart app as an example. These could be our tests:

  • Unit tests:

    a) Passing a cart object, the AddToCart component renders an Add to cart link if the product is not in the array.

    b) Passing a cart object, the AddToCart component renders a "View cart" link if the product is in the array.

  • Integration tests:

    a) Go to a product page and click "Add to cart." The link changes to "View cart."

    b) Go to the checkout page. After clicking on the Checkout button, it gets disabled.

  • One E2E test testing the cart flow:

    a) Go to a product page, click Add to cart, then click on View cart.

    b) You should have got to the checkout page. Click Checkout.

    c) You should have been redirected to the receipt page.

    d) The receipt should show the product added to the cart.

    e) The price should be the product price.

We are at the top of the pyramid. That means that these will be the slowest and least stable tests.

Why least stable? Check the workflow. Many bad things can happen there. The add to cart endpoint might take a little bit more than expected. The scroll to the Checkout button could have failed for just a few pixels. Your database might be in an unexpected state. Maybe your user already purchased that product, so the Add to cart button is not enabled.

How about roles?

This is the QA Analyst's land. This is where they need to take advantage of all the features Puppeteer provides to make reliable tests. But Developers play an important role, helping the QA team to do their job efficiently. As we are going to see in the next chapters, a developer can leave hints so that the QA team can find the components they need.

I hope the picture of the pyramid makes more sense now. We need lots of small and isolated unit tests, many integration tests testing our pages, and finally, a good set of E2E tests, checking the workflow's health.

This is the famous testing pyramid, but how do we write a test? Where do we write them? How do we run a test?

First, we need to know what we need from a test runner.

 

Test runner features

What would the world be like without a test runner? Let's say you don't know what a test runner is, and you want to code a unit test. Would that be possible? I think it would. For instance, say we have this small Cart class:

class Cart {
    constructor() {
        this._cart = [];
    }
    total() {
        return this._cart.reduce((acc, v) => acc + v.price, 0);
    };
    addToCart(item) {
        this._cart.push(item);
    };
}
module.exports = Cart;

If we want to test it, we could run some code like this:

const Cart = require('./cart.js');
const c = new Cart();
c.addToCart({ productId: 10, price: 5.5});
c.addToCart({ productId: 15, price: 6.5});
if(c.total() !== 12)
    console.error('Nooo!!!');
else
    console.log('Yes!!!!!');
 

A test is basically a piece of code testing our code. Will this work? Yes. Is this a unit test? Yes. Will this scale? Definitely not. This file will become massive and hard to maintain. Keeping track of what has failed would be an impossible task. We need a tool to help us scale and to help us keep our tests maintainable. We need a test runner.

Before exploring possible test runners, I would like to review what we would expect from a test runner. What are the features we would need in a test runner?

Easy to learn and run

We have a lot of things to learn. We need to learn Node and React; we even have to buy a book about Puppeteer. We want a test runner that is simple and easy to use.

Group tests by functionality

We want to have our tests separated by functionality, component, or workflow. Most test runners have a describe function that helps us to group tests.

Ignore tests if needed

We want to skip a test if it becomes noisy, but we don't want to remove it.

Run only one test

Being able to run only one test is extremely important while debugging. Imagine you have over 1,000 tests (yes, you are going to have over 1,000 tests). If you want to fix only one test, you wouldn't want to run all of them. You would like to run only the one you are working on.

Assertions

Assertions are essential. An assertion is an expression to check whether the program we are testing worked as expected. Do you remember my console.log and console.error to check whether the cart worked as expected? Well, Assertions are way better than that. What do we want to check with Assertions? This is a possible list:

  • Whether a value is equal to a test value.
  • Whether a value is null or not null.
  • Whether a string or a list contains a value. We might have a huge block of text, and we only want to check whether it has some string in it, or an item in an array.
  • Whether we expected something to fail, because sometimes, we would expect some piece of code to fail.

Tools to set up and clean up the environment

Before starting the tests, we need our application to be in a certain state. For instance, in the cart test, we would like to make sure that the customer has not already purchased the product before starting the test.

There are also technical setups that might need to be performed. In our case, we would need to have Puppeteer and a browser ready to be used before each test.

Another important concept is that tests should be independent and detached from each other. This means that the result of one test must not affect other tests. This is why, very often, it is required to clean up after each or all tests.

Reports

We want to see which tests passed and which tests failed. We would expect a test runner to at least show a good report in the terminal. It could be even better if we can get results in other formats, such as JSON, XML, or HTML.

There are many other features we could mention, but these are the most important features we need to know about before getting started.

Let's now see what the test runners available on the market that can cover the features we are requesting.

 

Available test runners

There are many types of tennis racquets. Some racquets give you more control. Others give you more power. If you have just started learning how to play tennis, you won't feel any difference. You would if you compared a cheap racquet with a professional one. But you wouldn't be able to say why one is better than the other. You would say that it just feels better.

It's the same with test runners. There are test runners that offer some features. Other runners offer other features. But what's important for us now is to get a test runner that provides us with all the required features to write our automated tests.

Another important thing to mention is that this book is not about "using Puppeteer with X." We are going to pick a test runner after this chapter, but it doesn't need to be the test runner for you. The idea is that you can choose what's best for you, or what your team is using right now. It is also probable that by the time you read this book, a better test runner will have become popular. You should be able to apply the concepts you learned from this book to that test runner.

These are the most common test runners in the market today.

Jest

According to the Jest site (https://jestjs.io/), "Jest is a delightful JavaScript Testing Framework with a focus on simplicity." Pretty nice introduction. Facebook maintains this project, and it currently has over 32,000 stars on GitHub. I'm not saying this is what makes a project a good project, but knowing who is behind a project and its level of community support are some of the things to take into consideration.

Jest has all the features we mentioned before, such as group tests with describe, and each test is an it or test function. You can skip tests with describe.skip, it.skip, or test.skip. You can run only one test with describe.only, it.only, or test.only. You also have beforeEach, afterEach, beforeAll, and afterAll, to run setup and cleanup code.

It also has some features that differentiate it from other runners. It has a Snapshot tool. The snapshot tool would process a React component and return some kind of DOM representation as JSON, which will allow us to test whether the DOM created by the component has changed. Is this a kind of UI test? Sure it is!

Another thing to consider when evaluating a test runner is available plugins. For instance, there is a package called jest-puppeteer, which helps us integrate our tests with Puppeteer. You don't need to use jest-puppeteer. It's just a helper.

There is also a package called jest-image-snapshot, maintained by American Express, which provides a set of tools to perform visual regression tests. In this case, if you want to code visual regression tests, I recommend you to use one of these packages. Managing all the screenshot baselines can be quite tedious.

Mocha

Mocha is another popular framework. It is a community project with over 19,000 stars. Something worth mentioning is that the Puppeteer team uses Mocha.

Mocha also has functions like Jest. It has a describe function to group tests. Tests are it functions. You can skip functions using describe.skip or it.skip, and use describe.only or it.only to run only one test. You also have beforeEach, afterEach, beforeAll, and afterAll, to run setup and cleanup code.

You will also find many plugins for Mocha. You will find mocha-puppeteer and mocha-snapshots.

A recipe you are going to see a lot on the web is Mocha + Chai. Chai is an assertion library that extends the assertions a test runner provides. It lets you express assertions in a pretty specific way:

foo.should.be.a('string');
foo.should.equal('bar');
foo.should.have.lengthOf(3);
tea.should.have.property('flavors').with.lengthOf(3);

There are many other test runners, such as Jasmine by Pivotal Labs with over 15,000 stars, Karma by the AngularJS team with over 11,000 stars, AVA, a community project with over 18,000 stars, and the list goes on.

As I mentioned at the beginning of this section, we just need a good tennis racquet, that is, a good test runner. When you become an expert, you will be able to move from one test runner to another that fits your needs. For the purpose of this book, we are going to use Mocha + Chai.

 

Creating our first test project

We will create a Node application in the same way we created our first app in Chapter 1, Getting started with Puppeteer. We are going to create a folder called OurFirstTestProject (you will find this directory inside the Chapter2 directory mentioned in the Technical requirements section) and then execute npm init -y inside that folder:

> npm init -y

The response should be something like this:

{
  "name": "OurFirstTestProject",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Now it's time to install the packages we are going to use:

  • Puppeteer 7
  • Mocha (any version)
  • Chai (any version)

Let's run the following commands:

> npm install puppeteer@">=7.0.0 <8.0.0"
> npm install mocha
> npm install chai

For this first demo, we are going to use the site https://www.packtpub.com/ as a test case. Let's keep our test simple. We want to test that the page title says Packt | Programming Books, eBooks & Videos for Developers.

Important Note

The site we are using for this test might have changed over time. Before testing this code, go to https://www.packtpub.com/ and check whether the title is still the same. That's why, in the following chapters, we will be downloading sites locally, so we avoid these possible issues.

We mentioned that we would use describe to group our tests. But separating tests into different files will also help us to get our code organized. You can choose between having one or many describe functions per file. Let's create a file called home.tests.js. We are going to put all tests related to the home page there.

Although you can create the files anywhere you want, Mocha grabs all the tests in the test folder by default, so we will to create the test folder and then create the home.test.js file inside that folder.

We are going to have the following:

  • home.tests.js with the home tests
  • A describe function with the header tests
  • An it function testing "Title should have Packt name"
  • Another it function testing "Title mention the word Books"

The structure should look like this:

const puppeteer = require('puppeteer');
const expect = require('chai').expect;
const should = require('chai').should();
describe('Home page header', () => {
    it('Title should have Packt name', async() => {
    });
    it('Title should mention Books', async() => {
    });
});

Let's unpack this code:

  1. We are importing Puppeteer in line 1.
  2. Lines 2 are 3 are about importing the different types of assertion styles Chai provides. As you can see, expect is not being called with parentheses whereas should is. We don't need to know why now. But, just to be clear, that's not a mistake.
  3. How about Mocha? Are we missing Mocha? Well, Mocha is the test runner. It will be the executable we will call later in package.json. We don't need it in our code.
  4. It's interesting to see that both describe and it are just simple functions that take two arguments: a string and a function. Can you pass a function as an argument? Yes, you can!
  5. The functions we are passing to the it functions are async. We can't use the await keyword in functions that are not marked as async. Remember that Puppeteer relies a lot on async programming.

Now we need to launch a browser and set up everything these tests need to work. We could do something like this:

it('Title should have Packt name', async() => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://www.packtpub.com/');
    // Our test code
    await browser.close();
});

Tip

Don't try to learn the Puppeteer API now. We are going to explain how all of these commands work in Chapter 3, Navigating through a website.

This code will run perfectly. However, there are two things that could do with optimization:

  • We would be repeating the same code over and over.
  • If something fails in the middle of the test, the browser won't get closed, leaving lots of open browsers.

To avoid these problems, we can use before, after, beforeEach, and afterEach. If we add these functions to our tests, this would be the execution order:

  • before
  • beforeEach
  • it('Title should have Packt name')
  • afterEach
  • beforeEach
  • it('Title should mention Books')
  • afterEach
  • after

It's not a rule of thumb, but we can do something like this in our case:

  1. before: Launch the browser.
  2. beforeEach: Open a page and navigate to the URL.
  3. Run the test.
  4. afterEach: Close the page.
  5. after: Close the browser.

These hooks, which is what Mocha calls these functions, would look like this:

let browser;
let page;
before(async () => {
    browser = await puppeteer.launch();
});
beforeEach(async () => {
    page = await browser.newPage();
    await page.goto('https://www.packtpub.com/');
});
afterEach(async () => {
    await page.close();
});
after(async () => {
    await browser.close();
});

One thing to mention here is that we could do what's called Fire and Forget when closing the page or the browser. Fire and forget means that we don't want to await the result of page.close() or browser.close(). So, we could do this:

afterEach(() => page.close());
after(() => browser.close());

That's not something I love doing because if something fails, you would like to know where and why. But as this is just cleanup code for a test, it's not production code, we can afford that risk.

Now our test has a browser opened, a page with the URL we want to test read. We just need to test the title:

it('Title should have Packt name', async() => {
    const title = await page.title();
    title.should.contain('Packt');
});
it('Title should should mention Books', async() => {
    expect((await page.title())).to.contain('Books');
});

I used two different styles here.

In the first case, I'm assigning the result of the title async function to a variable, and then using should.contain to check whether the title contains the word "Packt". In the second case, I just evaluated ((await page.title()). I added some extra parentheses there for clarification. You won't see them in the final example.

The second difference is that in the first case, I'm using the should style, whereas in the second case, I'm using the expect style. The result will be the same. It's just about which style you feel more comfortable with or feels more natural to you. There is even a third style: assert.

We have everything we need to run our tests. Remember how npm init created a package.json file for us? It's time to use it. Let's set the test command. You should have something like this:

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1"
},

We need to tell npm to run Mocha when we execute npm test:

"scripts": {
  "test": "mocha"
},

Time to run our tests! Let's run npm test in the terminal:

npm test

And we should have our first error:

  1) Home page header
       "before each" hook for "Title should have Packt name":
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

That's bad, but not that bad. Mocha validates by default that our tests should take less than 2,000 ms. That sounds OK for an isolated unit test. But UI tests might take longer than 2 seconds. That doesn't mean that UI tests shouldn't have a timeout. Speed is a feature, so we should be able to enforce some expected timeout. We can change that by adding the --timeout command-line argument to the launch setting we set up in the package.config file. I think 30 seconds could be a reasonable timeout. As it expects the value in milliseconds, it should be 30000. Let's make that change in our package.config file:

"scripts": {
  "test": "mocha --timeout 30000"
},

Tip

The command-line argument is not the only way to set up the timeout. You can call this.Timeout (30000) inside the describe function or configure the timeout using a config file (https://mochajs.org/#configuring-mocha-nodejs).

Once we set up the timeout, we can try our tests again by running npm test:

Test Result

Test Result

Mocha not only ran our tests but also printed a pretty decent report. We have there all the tests Mocha ran, the final result, and the elapsed time. Here is where many test runners offer different options. For instance, Mocha has a --reporter flag. If you go to https://mochajs.org/, you will see all the available reporters. We could use the list reporter, which shows the elapsed time of each test. We can add it to our package.config file:

"scripts": {
  "test": "mocha --timeout 30000 --reporter=list"
},

With this change, we can get a better report:

Test Result using the list reporter

Test Result using the list reporter

This project looks fine. If you had only a few tests, this would be enough. But if we are going to have lots of tests using many pages, this code won't scale. We need to organize our code so that we can be more productive and reuse more code.

 

Organizing our code

Our first test was quite simple: we were just checking the page title. But let's take a look at the home page:

Packtpub home page

Packtpub home page

There are many actions we would like to test there:

  • Search for an existing book.
  • Search for a non-existing book.
  • Check the cart when it is empty.
  • Check the cart when we add a product.

Let's take, for example, Search tests. We would be doing the same steps every time:

  1. Click on the search box.
  2. Enter the text.
  3. Click on the search button.

We would be doing the same thing over and over in all our search tests. Sometimes there is a misconception that, as the test code is not production code, the code can be a mess. So, people go and copy/paste their tests over and over, duplicating code and hardcoding values. That ends up with hard-to-maintain tests. When tests are hard to maintain, they tend to be pushed down the priority list. Developers lose, QA analysts lose, and in the end, clients lose.

We are going to see two techniques to improve our test code: the Page Object Model (POM) and the test data config.

Introducing the Page Object Model

The POM is a design pattern that will help us separate our test code from the implementation of the interaction our tests will perform.

Let's build our HomePageModel together. What are the possible interactions on that page?

  • Go (to the page)
  • Get page title
  • Search
  • Sign In
  • View Cart
  • Go to Checkout
  • Subscribe

Well done! We just created our first Page Model. This is how it will look:

module.exports = class HomePageModel {
    go() {}
    title() {}
    search(searchValue) {}
    signIn() {}
    viewCart(){}
    gotoCheckout(){}
    subscribe(){}
}

Let's focus on the two first functions: the go function, which will navigate to the home page, and the title function, which will return the page title.

We will reuse a lot of code here. If we want to start using this model, we would need to do two things: implement the title fetching here and pass a Puppeteer page to this model:

export default class HomePageModel {
    constructor(page) {
        this.page = page;
    }
    // Unused functions…
    async go() {
        await this.page.goto('https://www.packtpub.com/');
    }
    async title() {
        return await this.page.title();
    }
}

Now it's a matter of importing this class into our tests using require. I will put this class into a POM (Page Object Model) folder inside the test folder. Once we create the file, we import it:

const HomePageModel = require('./pom/HomePageModel.js');

We declare a variable inside the describe:

let homePageModel;

We create an instance of this class in the beforeEach hook:

beforeEach(async () => {
    page = await browser.newPage();
    homePageModel = new HomePageModel(page);
    await homePageModel.go();
});

And now, we simply replace the page.title we are using with homePageModel.title:

(await homePageModel.title()).should.contain('Packt');

As I mentioned earlier in the chapter, UI tests help us see whether our refactoring broke our code. Let's run npm test again to confirm that we didn't break anything:

Test result after the first refactor

Test result after the first refactor

There's only one thing left to do so that we can be proud of our first project. We need to get rid of our hardcoded values. We only wrote two tests, and we have three hardcoded values: the site URL and the Packt and the Books words.

For these tests, we can leave these hardcoded values. But what if you have different environments? You would need to make the URL dynamic. What if your site were a generic e-commerce site? The brand name would depend on the test you are navigating.

There are many other use cases:

  • Test users and passwords
  • Product to test
  • Keywords to use

We can create a config.js file with all the environment settings and return only the one we get on an environment variable. If not set, we return the local version:

module.exports = ({
    local: {
        baseURL: 'https://www.packtpub.com/',
        brandName: 'Packt',
        mainProductName: 'Books'
    },
    test: {},
    prod: {},
})[process.env.TESTENV || 'local']

If this looks a little bit scary, don't worry, it's not that complex:

  • It returns an object with three properties: local, test, and prod.
  • In JavaScript, you can access a property by using object.property or by treating the object as a dictionary: object['local'].
  • process.env allows us to read environment variables. We won't be using environment variables in this book, but I wanted to show you the final solution.
  • Finally, we are going to return only the local, test, or prod property based on the TESTENV variable or 'local' if the environment variable was not set.

I bet that by now, you will know that we will be able to access this object using a require call:

const config = require('./config');

And from there, start using the config variable instead of hardcoded values. We would also need to pass this config to the page model because we have a hardcoded URL there.

After making all these changes, this is what our tests should look like:

const puppeteer = require('puppeteer');
const expect = require('chai').expect;
const should = require('chai').should();
const HomePageModel = require('./pom/HomePageModel.js');
const config = require('./config');
describe('Home page header', () => {
    let browser;
    let page;
    let homePageModel;
    before(async () => browser = await puppeteer.launch());
    beforeEach(async () => {
        page = await browser.newPage();
        homePageModel = new HomePageModel(page, config);
        await homePageModel.go();
    });
    afterEach(() => page.close());
    after(() => browser.close());
    it('Title should have Packt name', async() => {
        (await homePageModel.title()).should.contain(config.brandName);
    });
    it('Title should mention Books', async() => {
        expect(await homePageModel.title()).to.contain(config.mainProductName);
    });
});

If we remove all the unused functions, our final page model would look like this:

module.exports = class HomePageModel {
    constructor(page, config) {
        this.page = page;
        this.config = config;
    }
    async go() {
        await this.page.goto(this.config.baseURL);
    }
    async title() {
        return await this.page.title();
    }
}

As you can see, we didn't need to implement complex design patterns to make our tests reusable and easy to maintain. I think it's time to get started with our tests, which we will do in Chapter 3, Navigating through a website.

 

Summary

In this chapter, we started with the foundations of automated testing. Mike Cohn's pyramid helped us to understand the different types of tests. We also gave this pyramid a new look, showing how it should be used from a Frontend developer perspective. We also made it clear that both developers and QA analysts are part of this pyramid, but with different perspectives.

In the second part of the chapter, we got more practical, and we looked into test runners. A learning point here is that we used Mocha as a test runner, but everything you learned in this chapter should be possible with any test runner; that is, we used Mocha, but we could have used any other test runner.

We use many Puppeteer APIs in our tests. In the next chapter, we are going to dive deep into these APIs and see how we can use Puppeteer in different scenarios.

About the Author
  • Dario Kondratiuk

    Dario Kondratiuk is a web developer since 2001. He won the Microsoft MVP (most valuable professional) award in 2020 for his contributions to the developer community. Dario has been working with Puppeteer since the beta versions, back in 2017. He is the author of Puppeteer-Sharp, a Puppeteer port to .NET, and Playwright-Sharp, a Playwright port to .NET.

    Browse publications by this author
UI Testing with Puppeteer
Unlock this book and the full library FREE for 7 days
Start now