Chapter 10.  Working with Asynchronous Code, Testing, and Deploying an API with Tornado

In this chapter, we will take advantage of the non-blocking features combined with asynchronous operations in Tornado in a new version for the API we built in the previous chapter. We will configure, write, and execute unit tests and learn a few things related to deployment. We will cover  the following topics:

• Understanding synchronous and asynchronous execution

• Working with asynchronous code

• Refactoring code to take advantage of asynchronous decorators

• Mapping URL patterns to asynchronous and non-blocking request handlers

• Making HTTP requests to the Tornado non-blocking API

• Setting up unit tests

• Writing a first round of unit tests

• Running unit tests with nose2 and checking testing coverage

• Improving testing coverage

Understanding synchronous and asynchronous execution

In our current version of the API, each HTTP request is blocking, as happened with Django and Flask. Thus, whenever the Tornado HTTP server receives an HTTP request, it doesn't start working on any other HTTP request in the incoming queue until the server sends the response for the first HTTP request it received. The methods we coded in the request handlers are working with a synchronous execution and they don't take advantage of the non-blocking features included in Tornado when combined with asynchronous executions.

In order to set the brightness level for both the blue and white LEDs, we have to make two HTTP PATCH requests. We will make them to understand how our current version of the API processes two incoming requests.

Open two Cygwin terminals in Windows or two Terminals in macOS or Linux, and write the following command in the first one. We will compose and send an HTTP request to set the brightness level for the blue LED to 255...

Refactoring code to take advantage of asynchronous decorators

It is extremely difficult to read and understand code split into different methods, such as the asynchronous code that requires working with callbacks that are executed once the asynchronous execution finishes. Luckily, Tornado provides a generator-based interface that enables us to write asynchronous code in request handlers in a single generator. We can avoid splitting our methods into multiple methods with callbacks by using the tornado.gen generator-based interface that Tornado provides to make it easier to work in an asynchronous environment.

The recommended way to write asynchronous code in Tornado is to use coroutines. Thus, we will refactor our existing code to use the @tornado.gen.coroutine decorator for asynchronous generators in the required methods that process the different HTTP requests in the subclasses of tornado.web.RequestHandler.

Tip

Instead of working with a chain of callbacks, coroutines use the Python yield...

Mapping URL patterns to asynchronous request handlers

We must map URL patterns to our previously coded subclasses of tornado.web.RequestHandler that provide us asynchronous methods for our request handlers. The following lines create the main entry point for the application, initialize it with the URL patterns for the API, and start listening for requests. Open the previously created async_api.py file and add the following lines. The code file for the sample is included in the restful_python_chapter_10_01 folder:

application = web.Application([ 
    (r"/hexacopters/([0-9]+)", AsyncHexacopterHandler), 
    (r"/leds/([0-9]+)", AsyncLedHandler), 
    (r"/altimeters/([0-9]+)", AsyncAltimeterHandler), 
],debug=True) 
 
 
if __name__ == "__main__": 
    port = 8888 
    print("Listening at port {0}".format(port)) 
    application.listen(port) 
    ioloop.IOLoop.instance().start() 

The code creates an instance of tornado.web.Application named application with the collection of request handlers that...

Making HTTP requests to the Tornado non-blocking API

Now, we can run the async_api.py script that launches Tornados's development server to compose and send HTTP requests to our new version of the Web API that uses the non-blocking features of Tornado combined with asynchronous execution. Execute the following command:

python async_api.py

The following lines show the output after we execute the previous command. The Tornado HTTP development server is listening at port 8888:

Listening at port 8888

With the previous command, we will start the Tornado HTTP server and it will listen on every interface on port 8888. Thus, if we want to make HTTP requests to our API from other computers or devices connected to our LAN, we don't need any additional configurations.

In our new version of the API, each HTTP request is non-blocking. Thus, whenever the Tornado HTTP server receives an HTTP request and makes an asynchronous call, it is able to start working on any other HTTP request in the incoming queue...

Setting up unit tests

We will use nose2 to make it easier to discover and run unit tests. We will measure test coverage, and therefore, we will install the necessary package to allow us to run coverage with nose2. First, we will install the nose2 and cov-core packages in our virtual environment. The cov-core package will allow us to measure test coverage with nose2.

Make sure you quit the Tornado's HTTP server. Remember that you just need to press Ctrl + C in the Terminal or command-prompt window in which it is running. We just need to run the following command to install the nose2 package that will also install the six dependency:

pip install nose2

The last lines for the output will indicate that the nose2 package has been successfully installed:

    
    Collecting nose2
    Collecting six>=1.1 (from nose2)
      Downloading six-1.10.0-py2.py3-none-any.whl
    Installing collected packages: six, nose2
    Successfully installed nose2-0.6.5 six-1.10.0

We just need to run the following...

Writing a first round of unit tests

Now, we will write a first round of unit tests. Specifically, we will write unit tests related to the LED resources. Create a new tests subfolder within the virtual environment's root folder. Then, create a new test_hexacopter.py file within the new tests subfolder. Add the following lines that declare many import statements and the TextHexacopter class. The code file for the sample is included in the restful_python_chapter_10_02 folder:

import unittest 
import status 
import json 
from tornado import ioloop, escape 
from tornado.testing import AsyncHTTPTestCase, gen_test, gen 
from async_api import Application 
 
 
class TestHexacopter(AsyncHTTPTestCase): 
    def get_app(self): 
        self.app = Application(debug=False) 
        return self.app 
 
    def test_set_and_get_led_brightness_level(self): 
        """ 
        Ensure we can set and get the brightness levels for both LEDs 
        """ 
        patch_args_led_1 = {'brightness_level': 128} ...

Running unit tests with nose2 and checking testing coverage

Now, run the following command to create all the necessary tables in our test database and use the nose2 test running to execute all the tests we created. The test runner will execute all the methods for our TestHexacopter class that start with the test_ prefix and will display the results. In this case, we just have one method that matches the criteria, but we will add more later.

Run the following command within the same virtual environment we have been using. We will use the -v option to instruct nose2 to print test case names and statuses. The --with-coverage option turns on test coverage reporting generation:

nose2 -v --with-coverage

The following lines show the sample output. Notice that the numbers shown in the report might have small differences if our code includes additional lines or comments:

test_set_and_get_led_brightness_level (test_hexacopter.TestHexacopter) ... 
I've started setting the Blue LED's brightness level...

Improving testing coverage

Now, we will write additional unit tests to improve the testing coverage. Specifically, we will write unit tests related to the hexacopter motor and the altimeter. Open the existing test_hexacopter.py file and insert the following lines after the last line. The code file for the sample is included in the restful_python_chapter_10_03 folder:

    def test_set_and_get_hexacopter_motor_speed(self): 
        """ 
        Ensure we can set and get the hexacopter's motor speed 
        """ 
        patch_args = {'motor_speed': 700} 
        patch_response = self.fetch( 
            '/hexacopters/1',  
            method='PATCH',  
            body=json.dumps(patch_args)) 
        self.assertEqual(patch_response.code, status.HTTP_200_OK) 
        get_response = self.fetch( 
            '/hexacopters/1', 
            method='GET') 
        self.assertEqual(get_response.code, status.HTTP_200_OK) 
        get_response_data = escape.json_decode(get_response.body) 
        self...

Other Python Web frameworks for building RESTful APIs

We built RESTful Web Services with Django, Flask, and Tornado. However, Python has many other Web frameworks that are also suitable for building RESTful APIs. Everything we learned throughout the book about designing, building, testing, and deploying a RESTful API is also applicable to any other Python Web framework we decide to use. The following list enumerates additional frameworks and their main Web page:

• Pyramid: http://www.pylonsproject.org/

• Bottle: http://bottlepy.org/

• Falcon: https://falconframework.org/

As always happens with any Python Web framework, there are additional packages that might simplify our most common tasks. For example, it is possible to use Ramses in combination with Pyramid to create RESTful APIs by working with RAML (RESTful API Modeling Language), whose specification is available at http://github.com/raml-org/raml-spec. You can read more details about Ramses at http://ramses.readthedocs.io/en/stable/getting_started...

Test your knowledge

1. The concurrent.futures.ThreadPoolExecutor class provides us:

   1. A high-level interface for synchronously executing callables.

   2. A high-level interface for asynchronously executing callables.

   3. A high-level interface for composing HTTP requests.

2. The @tornado.concurrent.run_on_executor decorator allows us to:

   1. Run an asynchronous method synchronously on an executor.

   2. Run an asynchronous method on an executor without generating a Future.

   3. Run a synchronous method asynchronously on an executor.

3. The recommended way to write asynchronous code in Tornado is to use:

   1. Coroutines.

   2. Chained callbacks.

   3. Subroutines.

4. The tornado.Testing.AsyncHTTPTestCase class represents:

   1. A test case that starts up a Flask HTTP Server.

   2. A test case that starts up a Tornado HTTP Server.

   3. A test case that doesn't start up any HTTP Server.

5. If we want to convert the bytes in a JSON response body to a Python dictionary, we can use the following function:

   1. tornado.escape.json_decode

   2. tornado.escape.byte_decode

   3. tornado.escape...

Summary

In this chapter, we understood the difference between synchronous and asynchronous execution. We created a new version of the RESTful API that takes advantage of the non-blocking features in Tornado combined with asynchronous execution. We improved scalability for our existing API and we made it possible to start executing other requests while waiting for the slow I/O operations with sensors and actuators. We avoided splitting our methods into multiple methods with callbacks by using the tornado.gen generator-based interface that Tornado provides to make it easier to work in an asynchronous environment.

Then, we set up a testing environment. We installed nose2 to make it easy to discover and execute unit tests. We wrote a first round of unit tests, measured test coverage, and then we wrote additional unit tests to improve test coverage. We created all the necessary tests to have a complete coverage of all the lines of code.

We built RESTful Web Services with Django, Flask, and Tornado...