Chapter 5. Writing End-to-End Tests

In the previous chapter, Chapter 4, Setting Up Development Tools, we successfully bootstrapped our project. In this chapter, we'll begin the development of our user directory API, which simply consists of Create, Read, Update, and Delete (CRUD) endpoints.

InChapter 1, The Importance of Good Code, we discussed the importance of testing and briefly outlined the principles and high-level processes ofTest-Driven Development(TDD). But theory and practice are two very different things. In this chapter, we will put the TDD approach into practice by first writingEnd-to-End(E2E) tests, and then using them to drive the development of our API. Specifically, we will do the following:

• Learn about different types of test
• Practice implementing a TDD workflow, specifically following the Red-Green-Refactor cycle
• Write E2E tests with Cucumber and Gherkin

 

 

Understanding different types of test

First, let's learn about the different types of tests and how they all fit into our project's workflow. The first thing to note is that some tests are more technically-focused, while others are more business-focused; some tests are only concerned with a very small part of the whole system, while others test the system as a whole. Here's a brief overview of the most common types of tests you'll encounter:

• Unit tests: These test the smallest testable parts of an application, called units. For example, if we have a function called createUser, we can write a unit test that tests that the function always returns a promise. With unit tests, we are only concerned with the function of the unit, independent of external dependencies. If the unit has external dependencies, such as a database, we must substitute the real database client with a fake one. This fake client must be able to mimic the behavior of the database adequately so that, from the perspective of...

Following a TDD workflow

Next, let's examine a typical TDD workflow, and see how the different types of tests fit into it.

Gathering business requirements

The TDD workflow starts with the product manager gathering business requirements from the business stakeholders, and then liaising with the technical team to refine these requirements, taking into account feasibility, costs, and time constraints.

The scope of the requirements should be small. If the application is large, the product manager should prioritize the requirements by importance and urgency, and group them into phases. The first phase should contain the highest priority requirements, which would be implemented first.

These requirements should be well-defined and unambiguous, so that there's no room for (mis)interpretation. This means they should quantified as much as possible. For example, instead of "the app must load quickly", it should say "the app must load within 1 second on an iPhone 5S".

Secondly, the requirement-gathering...

Gathering requirements

Now that we understand the workflow, let's put it into practice!

We begin byselecting a small portion of our application and defining its requirements. We picked the Create User feature because many other features depend on it. Specifically, the feature requires us to create an API endpoint, /users, that accepts POST requests, and stores the JSON payload of the request (representing the user) into a database. In addition, the following constraints should be applied:

• The user payload must include the email address and password fields
• The user payload may optionally provide a profile object; otherwise, an empty profile will be created for them

Now that we have our requirements, let's write our specification as E2E tests, using a tool called Cucumber.

Setting Up E2E tests with Cucumber

Cucumber is an automated test runner that executes tests written in a Domain-Specific Language (DSL) called Gherkin. Gherkin allows you to write tests in plain language, usually in a behavior-driven way, which can be read and understood by anyone, even if they are not technically-minded.

There are many Cucumber implementations for different languages and platforms, such as Ruby, Java, Python, C++, PHP, Groovy, Lua, Clojure, .NET and, of course, JavaScript. The JavaScript implementation is available as an npm package, so let's add it to our project:

$ yarn add cucumber --dev

We are now ready to write the specification for our first feature.

 

 

Features, scenarios, and steps

To use Cucumber, you'd first separate your platform into multiple features; then, within each feature, you'd define scenarios to test for. For us, we can take the "Create user" requirement as one feature, and start breaking it down into scenarios, starting with the following:

• If the client sends...

Implementing step definitions

To test our API server, we would need to run the server itself and send HTTP requests to it. There are many ways to send requests in Node.js:

• Using the request method provided by Node's native http module.
• Using the new Fetch Web API syntax: fetch is an improvement on the traditional XMLHttpRequest used to make AJAX (Asynchronous JavaScript And XML) requests from the client. We can use polyfills, such as isomorphic-fetch (https://www.npmjs.com/package/isomorphic-fetch), which will allow us to use the same syntax on the server.
• Using a library, such as request (https://www.npmjs.com/package/request), superagent (npmjs.com/package/superagent), axios (npmjs.com/package/axios), and many more.

Using the native http module allows us to be as expressive as possible because it works at the lowest-level API layer; however, this also means the code is likely to be verbose. Using the Fetch API might provide a simpler syntax, but it will still have a lot of boilerplate code...

Validating data type

We have completed our first scenario, so let's move on to our second and third scenarios. As a reminder, they are as follows:

• If the client sends a POST request to /users with a payload that is not JSON, our API should respond with a 415 Unsupported Media Type HTTP status code and a JSON object payload containing an appropriate error message.
• If the client sends a POST request to /users with a malformed JSON payload, our API should respond with a 400 Bad Request HTTP status code and a JSON response payload containing an appropriate error message.

 

Start by adding the following scenario definition to the spec/cucumber/features/users/create/main.feature file:

  Scenario: Payload using Unsupported Media Type

  If the client sends a POST request to /users with an payload that is 
  not JSON,
  it should receive a response with a 415 Unsupported Media Type HTTP 
  status code.

    When the client creates a POST request to /users
    And attaches a generic non-JSON payload
...

Migrating our API to Express

There are two ways to install Express: directly in the code itself or through the express-generator application generator tool. The express-generator tool installs the express CLI, which we can use to generate an application skeleton from. However, we won't be using that because it's mainly meant for client-facing applications, while we are just trying to build a server-side API at the moment. Instead, we'll add the express package directly into our code.

First, add the package into our project:

$ yarn add express

Now open up your src/index.js file, and replace our import of the http module with the express package. Also replace the current http.createServer and server.listen calls with express and app.listen. What was previously this:

...
import http from 'http';
...
const server = http.createServer(requestHandler);
server.listen(8080);

Would now be this:

...
import express from 'express';
...
const app = express();
app.listen(process.env.SERVER_PORT);

To help us know...

Moving common logic into middleware

Let's see how we can improve our code further. If you examine our Create User endpoint handler, you may notice that its logic could be applied to all requests. For example, if a request comes in carrying a payload, we expect the value of its Content-Type header to include the string application/json, regardless of which endpoint it is hitting. Therefore, we should pull that piece of logic out into middleware functions to maximize reusability. Specifically, these middleware should perform the following checks:

• If a request uses the method POST, PUT or PATCH, it must carry a non-empty payload.
• If a request contains a non-empty payload, it should have its Content-Type header set. If it doesn't, respond with the 400 Bad Request status code.
• If a request has set its Content-Type header, it must contain the string application/json. If it doesn't, respond with the 415 Unsupported Media Type status code.

 

Let's translate these criteria into Cucumber/Gherkin specifications...

Validating our payload

So far, we've been writing tests that ensure our request is valid and well-formed; in other words, making sure they are syntactically correct. Next, we are going to shift our focus to writing test cases that look at the payload object itself, ensuring that the payload has the correct structure and that it is semantically correct.

Checking for required fields

In our requirements, we specified that in order to create a user account, the client must provide at least the email and password fields. So, let's write a test for this.

 

 

In our spec/cucumber/features/users/create/main.feature file, add the following scenario outline:

Scenario Outline: Bad Request Payload

  When the client creates a POST request to /users
  And attaches a Create User payload which is missing the <missingFields> field
  And sends the request
  Then our API should respond with a 400 HTTP status code
  And the payload of the response should be a JSON object
  And contains a message property which...

Testing the success scenario

We have covered almost all of the edge cases. Now, we must implement the happy path scenario, where our endpoint is called as intended, and where we are actually creating the user and storing it in our database.

Let's carry on with the same process and start by defining a scenario:

Scenario: Minimal Valid User

  When the client creates a POST request to /users
  And attaches a valid Create User payload
  And sends the request
  Then our API should respond with a 201 HTTP status code
  And the payload of the response should be a string
  And the payload object should be added to the database, grouped under the "user" type

All steps are defined except the second, fifth, and last step. The second step can be implemented by using our getValidPayload method to get a valid payload, like so:

When(/^attaches a valid (.+) payload$/, function (payloadType) {
  this.requestPayload = getValidPayload(payloadType);
  this.request
    .send(JSON.stringify(this.requestPayload)...

Summary

In this chapter, we coerced you into following TDD principles when developing your application. We used Cucumber and Gherkin to write our end-to-end test, and used that to drive the implementation of our first endpoint. As part of our refactoring efforts, we've also migrated our API to use the Express framework.

At this point, you should have the TDD process drilled into your brain: Red. Green. Refactor. Begin by writing out test scenarios, implementing any undefined steps, then run the tests and see them fail, and finally, implementing the application code to make them pass. Once the tests have passed, refactor where appropriate. Rinse and repeat.

It's important to remember that TDD is not required to have self-testing code. You can, without following TDD, still write tests after to verify behavior and catch bugs. The emphasis of TDD is that it translates the design of your system into a set of concrete requirements, and uses these requirements to drive your development. Testing is...