Python API Development Fundamentals

By Jack Chan , Ray Chung , Jack Huang
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. 1. Your First Step

About this book

Python is a flexible language that can be used for much more than just script development. By knowing the Python RESTful APIs work, you can build a powerful backend for web applications and mobile applications using Python.

You'll take your first steps by building a simple API and learning how the frontend web interface can communicate with the backend. You'll also learn how to serialize and deserialize objects using the marshmallow library. Then, you'll learn how to authenticate and authorize users using Flask-JWT. You'll also learn how to enhance your APIs by adding useful features, such as email, image upload, searching, and pagination. You'll wrap up the whole book by deploying your APIs to the cloud.

By the end of this book, you'll have the confidence and skill to leverage the power of RESTful APIs and Python to build efficient web applications.

Publication date:
November 2019
Publisher
Packt
Pages
372
ISBN
9781838983994

 

1. Your First Step

Learning Objectives

By the end of this chapter, you will be able to:

  • Replicate the concepts of RESTful API
  • Describe the meaning of different HTTP methods and statuses
  • Get hands-on experience on PyCharm IDE
  • Build a RESTful API and execute CRUD using Flask
  • Use JSON messages to communicate with the APIs
  • Test API endpoints using Postman and httpie/curl command-line tools

This chapter introduces API and explains the concepts of web services, API and REST.

 

Introduction

We are in the internet era, a world where everything is connected. Data flows seamlessly from one place to another. We can get all the information in the world with a few clicks on a website. Take Skyscanner as an example, we just need to put in the date and location of our trips, and it can find us the cheapest flight in a split second; the hero behind the scenes that provides this data is API.

In this chapter, you will learn what a web service, an API, and REST are. We will start by teaching the fundamental concepts of APIs. Then we will look at real-life examples of how different web services (Google, Facebook, and so on) use the REST API.

Finally, we will develop our first simple Python RESTful API using Python. Python is a popular and powerful programming language. Apart from its extensive use in the realm of artificial intelligence, it is also widely used in web application development, big data analysis, web scraping, and process automation. What makes Python excel in so many areas is the extensive number of frameworks available. The frameworks do all the heavy lifting jobs and that allows the developers to focus on the actual application design and development.

In this chapter, you will see how data is encoded and communicated between the frontend and the backend. You will learn technical details about the JSON format, the HTTP protocol, HTTP status codes, and so on. All the development work will be verified and tested using Postman and httpie/curl. We will take you through the whole process of web application development. Not only will you learn the essential aspects of developing a RESTful API, but you will also learn about the thinking process, design, development, testing, and even deployment. This is a journey of learning the complete software development life cycle. Let's embark on our exciting journey now!

 

Understanding API

API stands for application programming interface; it is an interface for the website (or mobile application) to communicate with the backend logic. Simply put, it is like a messenger that takes a request from the users and sends the request to the backend system. Once the backend system responds, it will then pass that response to the users. A metaphor for this is a waiter/waitress, who can understand different customers' orders. They will then act as a middleman between the customers and the chefs in the kitchen.

If you were the boss of the restaurant, the key benefit of having a waiter/waitress here between your customer and the kitchen is that the customers will be shielded from seeing your business secrets. They don't need to know how the meal is prepared. They just need to send an order through the waiter/waitress, and they will get the meal they ordered. In this scenario, the waiter acts like the API. The following figure helps illustrate the analogy.

Figure 1.1: The waiter acting as the API for the customer
Figure 1.1: The waiter acting as the API for the customer

Similarly, in computer science, one of the key benefits of having API is encapsulation. We encapsulate the logic so that people outside won't be able to see it. With this arrangement, big companies with sensitive information are willing to provide services to the world through APIs, confident that their internal information won't be revealed. Take Skyscanner again as an example. The company is comfortable with using an API to allow customers to book their flights, but at the same time, personal data from other customers that are stored in their internal database won't leak.

An API is also a standard interface that can communicate with different types of frontend Terminals, they can be mobile applications or websites. As long as the frontend is sending the same request to the API, it will get the same result back. If we go back to our metaphor, the waiter/waitress will serve all kinds of customers, regardless of their gender, age, language, and so on.

Now, imagine you are a software engineer at Skyscanner who is responsible for developing an API. What will your job be? Let me tell you. Your job will be to write a program that can take booking requests (date and location) from customers through the website, and then look up matching flights in the Skyscanner database and return the flight details to the customers. Throughout this book, you will be our API engineering intern. We will guide you, step by step, through the process of developing a RESTful API project that can serve the users of your system.

 

RESTful API

REST stands for Representational State Transfer. It was first defined in Dr. Roy Fielding's dissertation (Architectural Styles and the Design of Network-Based Software Architectures) back in 2000. This dissertation is considered to be the bible in the web domain. REST is not a standard or protocol; it is more like a software architectural style. Many engineers follow this architectural style to build their applications, such as eBay, Facebook, and Google Maps. These web applications serve huge amounts of traffic every second, so you can see that REST really is a scalable architecture style. And when we say RESTful API, we are referring to an API that conforms to the REST constraints/principles.

REST Constraints/Principles

There are five important constraints/principles for the REST architecture style:

  • Client-server: There is an interface between the client and the server. The client and server communicate through this interface and are independent of each other. Either side can be replaced as long as the interface stays the same. Requests always come from the client-side.
  • Stateless: There is no concept of state for a request. Every request is considered to be independent and complete. There is no dependence on the previous request nor dependence on a session to maintain the connection status.
  • Cacheable: Things are cacheable on the server or client-side to improve performance.
  • Layered system: There can be multiple layers in the system, and the goal here is to hide the actual logic/resources. These layers can perform different functions, such as caching and encryption.
  • Uniform interface: The interface stays the same. This helps to decouple the client and server logic.
 

HTTP Protocol

To better understand what REST is and make sure we are implementing the REST style, we can simply talk about the HTTP protocol. HTTP is an implementation of the REST architecture style. It is short for HyperText Transfer Protocol and is the standard protocol used on the worldwide web. We use it every day to browse different websites. That's why all the websites we visit are prefixed with http.

In the HTTP protocol, there are different types of service request methods. Each service request method has a special definition that is specific to it. When the frontend interface interacts with the backend API through a URL, they need to, at the same time, define the HTTP method for this request. Different HTTP methods are like different service counters. For example, reading and creating data are completely different services, so they should be handled by different service counters, meaning different HTTP methods.

  • GET: For reading data
  • POST: For creating data
  • PUT: For updating data by completely replacing data with new content
  • PATCH: For updating data, but by partially modifying a few attributes
  • DELETE: For deleting data

Simply put, different HTTP methods are like the verbs for REST API. They are used for performing different actions on the same set of data.

 

HTTP Methods and CRUD

We can easily build a RESTful API by leveraging what has already been provided by the HTTP protocol. Let's take a look at the HTTP methods that we can use to communicate with the server.

In this book, we will build a recipe sharing platform with a RESTful API as the backend. This platform will allow users to create and share their own recipes. At the same time, users will also be able to read recipes shared by other users. Using this recipe sharing platform as an example, to achieve these functionalities, we will need our API to be able to perform different actions on the recipes. We can leverage different HTTP methods here. For example, we can use the GET method to request http://localhost:5000/recipes for all the recipes. We can use the POST method to request http://localhost:5000/recipes to create a new recipe. We can also use the DELETE method to request http://localhost:5000/recipes/20 to delete a recipe with ID = 20. Please refer to the following table for details.

Figure 1.2: HTTP methods
Figure 1.2: HTTP methods

We can see that asking the backend API to work for us is simple. We can simply use the HTTP protocol to communicate our request.

In fact, with this recipe sharing platform, you can see the majority of the actions we require will revolve around CREATE, READ, UPDATE, and DELETE. This is generally true for all other web applications as well. In the developer community, we call this CRUD in short. In a nutshell, CRUD models the life cycle of database record management.

Modeling our web applications this way can help us easily construct a functioning web system, as these actions are related to the HTTP methods. Constructing our application with this architecture is simple, powerful, and highly readable.

As you can probably imagine, we will need to send information to the backend server. For example, you may want to store a recipe in the backend database. You send the recipe over HTTP with a pre-agreed format with the backend. A pre-agreed format can be understood as a language used to communicate with the waiter/waitress in our previous metaphor. In real life, we have different languages, such as English, German, Chinese, and so on. We need to speak the right language for the other side to understand. In the web API domain, there are two prevalent standards, JSON and XML. We will mainly talk about JSON here because it is more readable and widely adopted.

 

The JSON Format

JavaScript Object Notation (JSON) is a simple plaintext format that is capable of representing complex data structures. We can use this format to represent strings, numbers, arrays, and even objects. Once we have the information "JSONified," we can use this widely adopted format to communicate with the API.

We are going to show you what a JSON format file looks like. In the following example, you will see that we are representing two recipes in JSON format. A JSON document is a plaintext document; there is no encryption here. It is so readable that I am sure you can already tell (without further explanation) that there are two recipes here, each with an ID, name, and description.

Here are a few notes on JSON syntax:

  • Arrays are enclosed by []
  • Objects can be represented by {}
  • Names/values always exist in pairs, and are delimited by ":"
  • Strings are enclosed by ""

Following is a sample code file with JSON syntax:

{
  "recipes":[
    {
      "id":1,
      "name":"Egg Salad",
      "description":"Place an egg in a saucepan and..."
    },
    {
      "id":2,
      "name":"Tomato Pasta",
      "description":"Bring a large pot of lightly salted water to a boil..."
    }
  ]
}
 

HTTP Status Codes

An HTTP status code is a code that is returned in the HTTP protocol. It is usually hidden from users, so you probably didn't realize it exists. In fact, every HTTP response from the server contains a status code. And as we construct our RESTful API, we need to comply with the HTTP protocol. The status code helps the frontend client understand the status of their request, that is, whether it is a success or failure. For example, there could be a client request about creating a record in the backend database. In that case, once the database record has been successfully created, the server should return an HTTP status code 201 (Created). If there is an error (such as a syntax error in the JSON document), the server should return an HTTP status code 400 (Bad Request) instead.

Commonly used HTTP Status Codes

Let's discuss some commonly used status codes. They are as follows:

  • 200 OK means the request has been successful. The request could be a GET, PUT, or PATCH.
  • 201 Created means the POST request has been successful and a record has been created.
  • 204 No Content means the DELETE request has been successful.
  • 400 Bad Request means there is something wrong with the client request. For example, there is a syntax error in the JSON format.
  • 401 Unauthorized means the client request is missing authentication details.
  • 403 Forbidden means the requested resource is forbidden.
  • 404 Not Found means the requested resource doesn't exist.
 

Open API

Open API is a third-party API that is open to use. There are plenty of them available out there. Companies are eager to open their APIs to expand their user base but at the same time keep their source code proprietary. These APIs can be accessible by us as well. Let's take a look at some of the APIs from Facebook.

For example, we can use the HTTP GET method to access https://graph.facebook.com/{page_id}/feed, which will give us the feeds on the Facebook page with ID = {page_id}. We can send an HTTP request using the POST method to https://graph.facebook.com/{page_id}/feed, and then we can create a post on the Facebook page with ID = {page_id}.

Note

The Facebook fans page API details can be found at https://developers.facebook.com/docs/pages/publishing.

Now, let's look at another internet giant, Google. Google also provides some Gmail APIs that we can use to manage the email labels in our mailbox. Here is a screenshot from the Gmail API documentation:

Figure 1.3: Gmail API documentation
Figure 1.3: Gmail API documentation

Note

The Gmail Label API is available at https://developers.google.com/gmail/api/v1/reference/.

 

The Flask Web Framework

Flask is a web framework that we can use to easily build a web application. Web applications usually need some core functionalities, such as interacting with client requests, routing URLs to resources, rendering web pages, and interacting with backend databases. A web application framework such as Flask provides the necessary packages, modules that do the heavy lifting. So, as a developer, we only need to focus on the actual application logic.

There are, of course, other available web frameworks available on the market. One strong competitor of Flask is Django. It is also a Python web framework. The reason why we choose Flask in this book is that Flask is minimalistic. It is regarded as a micro-web-framework that only provides the absolutely essential packages for developers to start with. Because of that, it is easy to learn and is great for beginners.

And later, if we want to build further functions, there is a vast number of Flask extensions. You will see the power of Flask as we progress in this book.

 

Building a Simple Recipe Management Application

Let's do some simple exercises to test your knowledge. We are going to build a recipe-sharing platform throughout this book, and the API is the interface we expose to the public. We will first define what functions we want to provide and the corresponding URLs. These are the basic functions that we will probably need:

Figure 1.4: HTTP methods and functions
Figure 1.4: HTTP methods and functions

A typical recipe should have the following attributes

  • ID: The unique identifier for the recipe
  • Name: The name of the recipe
  • Description: The description of the recipe

We are going to build an API that lists all the recipes stored in our system. The API will be designed to return different results with different URLs. For example, http://localhost:5000/recipes is going to give us all the recipes stored in our system, while http://localhost:5000/recipes/20 will give us the recipe with ID = 20. Upon successful recipe retrieval, we will also see the HTTP status is set to 200 (OK). This indicates that our request has been successful.

When we create a new recipe, we use the HTTP POST method to query http://localhost:5000/recipes with all the necessary parameters to describe our recipe in JSON format. The JSON format is simply a key/value pair. If our request is successful, the recipe will be created in the backend and will return HTTP status 201 (Created). Together with the HTTP status, it will also send the recipe that has just been created in JSON format.

When we update a recipe, we use the HTTP PUT method to send the data to http://localhost:5000/recipes/20 with all the necessary parameters for the updated recipe in JSON format. If our request is successful, the recipe will be updated in the backend and it will return HTTP status 200 (OK). Together with the HTTP status, it will also send the updated recipe in JSON format.

When we delete a recipe, we can use the HTTP Delete method to send the data to http://localhost:5000/recipes/20. This will remove the recipe with ID = 20.

Now you know where we are heading to, let's roll up our sleeves and get our hands dirty!

Virtual Environment

It is always recommended for developers to develop their application inside a virtual environment instead of directly on their local environment.

The reason is that virtual environments are independent application development environments. We can create multiple virtual environments on a local machine, and these virtual environments can have their own version of Python, their own packages, their own environment variables, and so on. These virtual environments won't interfere with each other even though they are built on the same local machine.

In the following exercise, we will create a development project in the PyCharm IDE. We will show you how to set up a virtual environment for this project in PyCharm.

Exercise 1: Building Our First Flask Application

We are going to build our first Flask application in this exercise. You will realize how simple it is to build an application along the way. PyCharm is a great integrated development environment (IDE) with a nice GUI that will make our development process easier. We will learn about the workflow of application development, including the creation of the application project and installing the necessary Python packages:

  1. Create a new project in PyCharm with File > New Project. Name our project basic-api. PyCharm will automatically help us to create a virtual environment for this new project.
    Figure 1.5: Creating a new project
    Figure 1.5: Creating a new project

    It's a good practice for projects to run on their own assigned independent virtual environments, so these projects can run on different packages and they won't affect each other.

  2. Install the necessary packages in our virtual environment. To do that, we can create a file named requirements.txt in our project and type in the following text. We want to install Flask (version 1.0.3) and httpie (version 1.0.2):
    Flask==1.0.3
    httpie==1.0.2

    Following screenshot shows the installation of Flask and httpie in requirements.txt:

    Figure 1.6: Installing Flask and httpie in requirements.txt
    Figure 1.6: Installing Flask and httpie in requirements.txt

    PyCharm is going to prompt us on the missing package, as shown in the screenshot. Clicking on Install requirement will let PyCharm take care of the installation for us in the virtual environment. Once the installation is done, we can create our first Python file, called app.py.

    Note

    To install the Python packages, we can also run the pip install -r requirements.txt command in Terminal. It will yield the same result.

    The Flask package that we are installing is a web micro-framework. It is extremely lightweight and allows us to build a web service with just a few lines of code.

  3. Let's type in the following code in app.py, then right-click on the filename of app.py in the left panel, and select run app to execute our first web service in Flask:
    from flask import Flask
    app = Flask(__name__)
    @app.route("/")
    def hello():
        return "Hello World!"
    if __name__ == "__main__":
        app.run()

    What this does is it first imports the Flask package in app.py, then it instantiates a Flask object, and finally, it assigns it to the app variable. We have created the main function as the entry point for our startup script. This subsequently brings up the Flask web server. After that, we have defined our first API function, hello, which returns a "Hello World" response. Using the Flask decorator, we can route the GET request URL to this function.

  4. Now open the browser and type http://localhost:5000, You will see the string Hello World!. No special format, just plaintext. This means your first web service passed the test, it works!
    Figure 1.7: Browser showing Hello World in plaintext
Figure 1.7: Browser showing Hello World in plaintext

This is a very good start! Though this web service merely returns the plain text string, we can build a lot of stuff on top of that.

I hope you can see how simple it is to build a web service using Flask; it is literally just a few lines of code. In fact, there are more Flask extensions out there that can help us build fancy functions. And be patient, we will talk about that in the subsequent chapters. For now, let's stay simple and get ourselves familiar with Flask first.

For production-grade applications, data is usually stored in a database. We haven't looked at how to interact with the database yet, so for now, we are going to simply store them in memory. Since we are building a recipe sharing platform, we will be creating two recipes in our next exercise, and we'll let them live in the memory.

Exercise 2: Managing Recipes with Flask

In this exercise, we are going to work on our recipe management application with Flask. We will implement functionality to get recipes, to create recipes, and to update recipes. Without further ado, let's get started:

Note

For the complete code, please refer to https://github.com/TrainingByPackt/Python-API-Development-Fundamentals/tree/master/Lesson01/Exercise02.

  1. First, clean up app.py and start everything all over again, import the packages that we need for this web service from the preceding code:
    from flask import Flask, jsonify, request

    The jsonify package here is to convert our Python objects (such as a list) to JSON format. It will also change the content type in our HTTP response to application/json. Simply put, it takes care of the heavy lifting of converting to JSON format for us.

  2. Then we import the HTTPStatus enum, which includes different HTTP statuses:
    from http import HTTPStatus

    For instance, we will have HTTPStatus.CREATED (201) and HTTPStatus.NOT_FOUND (404).

  3. Create an instance of the Flask class
    app = Flask(__name__)
  4. Define the recipes list. We store two recipes in the list. They are stored in the memory
    recipes = [
        {
            'id': 1,
            'name': 'Egg Salad',
            'description': 'This is a lovely egg salad recipe.'
        },
        {
            'id': 2, 'name': 'Tomato Pasta',
            'description': 'This is a lovely tomato pasta recipe.'
        }
    ]
  5. Use the route decorator to tell Flask that the /recipes route will route to the get_recipes function, and the methods = ['GET'] argument to specify that the route decorator will only respond to GET requests:
    @app.route('/recipes', methods=['GET'])
    def get_recipes():

    Note

    Please note that if we don't specify methods argument, the default will still be only responding to GET requests.

  6. After that, use the jsonify function to convert the list of recipes to JSON format and respond to the client:
        return jsonify({'data': recipes})
  7. After getting a specific recipe, if you only want to retrieve one specific recipe, then use the /recipes/<int:recipe_id> route to trigger the get_recipe(recipe_id) function.
    @app.route('/recipes/<int:recipe_id>', methods=['GET'])

    The syntax <int:recipe_id> syntax means the value in the route will be assigned to the integer variable id integer variable and can be used in the function. Our function get_recipe(recipe_id) function will then loop through the whole "recipes" list and locate the recipe that has the id that we are looking for. If that recipe exists, then we will return it.

  8. Take a closer look at our get_recipe function. Get the next recipe in the loop by using recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None). Here, the line for recipe in recipes iterates through all the recipes in our recipe collection and finds out the recipe with id = recipe_id. Once we have found it, we store it in the iterator and retrieve it using the next function. If there is no such recipe with that ID, None will be returned:
    def get_recipe(recipe_id):
        recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None)
        if recipe:
            return jsonify(recipe)
        return jsonify({'message': 'recipe not found'}), HTTPStatus.NOT_FOUND
  9. Next, we will work on the create_recipe function, which creates a recipe in memory. Use the /recipes route to the create_recipe function and the "methods = [POST]" argument to specify that the route decorator will only respond to POST requests:
    @app.route('/recipes', methods=['POST'])
  10. After that, use the request.get_json method to get the name and description from the client POST request. These two values together with a self-incremented id that we generate will be stored in the recipe (dictionary object) and then appended to our recipes list. At this point in time, the recipe is created and stored:
    def create_recipe():
        data = request.get_json()
        name = data.get('name')
        description = data.get('description')
        recipe = {
            'id': len(recipes) + 1,
            'name': name,
            'description': description
        }
        recipes.append(recipe)
  11. Finally, return the recipe that has just been created in JSON format, together with an HTTP 201 (CREATED) status. The following code highlights this:
        return jsonify(recipe), HTTPStatus.CREATED 
  12. The next part of code is about updating recipes. Again, use the same line of code here, recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None) to get the recipe with a specific ID:
    @app.route('/recipes/<int:recipe_id>', methods=['PUT'])
    def update_recipe(recipe_id):
        recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None)
  13. The next few lines of code say that if we can't find the recipe, we will return a recipe not found message in JSON format, together with a HTTP NOT_FOUND status:
        if not recipe:
            return jsonify({'message': 'recipe not found'}), HTTPStatus.NOT_FOUND
  14. If we found the recipe, then perform the recipe.update function, and put in the new name and description you get from the client request:
        data = request.get_json()
        recipe.update(
            {
                'name': data.get('name'),
                'description': data.get('description')
            }
        )
  15. Finally, we convert the updated recipe to JSON format using the jsonify function and return together with a default HTTP status 200 (OK). The following code highlights this:
        return jsonify(recipe)
  16. The last few lines of code in our program is for starting up the Flask server:
    if __name__ == '__main__':
        app.run()
  17. Once the code is done, right-click on the app.py file and click run to start the application. The Flask server will be started up and our application is ready to be tested. The full code looks like this:
    from flask import Flask, jsonify, request
    from http import HTTPStatus
    app = Flask(__name__)
    recipes = [
        {
            'id': 1,
            'name': 'Egg Salad',
            'description': 'This is a lovely egg salad recipe.'
        },
        {
            'id': 2, 'name': 'Tomato Pasta',
            'description': 'This is a lovely tomato pasta recipe.'
        }
    ]
    @app.route('/recipes/', methods=['GET'])
    def get_recipes():
        return jsonify({'data': recipes})
    @app.route('/recipes/<int:recipe_id>', methods=['GET'])
    def get_recipe(recipe_id):
        recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None)
        if recipe:
            return jsonify(recipe)
        return jsonify({'message': 'recipe not found'}), HTTPStatus.NOT_FOUND
    @app.route('/recipes', methods=['POST'])
    def create_recipe():
        data = request.get_json()
        name = data.get('name')
        description = data.get('description')
        recipe = {
            'id': len(recipes) + 1,
            'name': name,
            'description': description
        }
        recipes.append(recipe)
        return jsonify(recipe), HTTPStatus.CREATED
    @app.route('/recipes/<int:recipe_id>', methods=['PUT'])
    def update_recipe(recipe_id):
        recipe = next((recipe for recipe in recipes if recipe['id'] == recipe_id), None)
        if not recipe:
            return jsonify({'message': 'recipe not found'}), HTTPStatus.NOT_FOUND 
        data = request.get_json()
        recipe.update(
            {
                'name': data.get('name'),
                'description': data.get('description')
            }
        )
        return jsonify(recipe)
    if __name__ == '__main__':
        app.run()

The output is shown in the following screenshot:

Figure 1.8: The final Flask server
Figure 1.8: The final Flask server

In the following sections, we will show you how to test your web service using curl/httpie or Postman.

 

Using curl or httpie to Test All the Endpoints

In this section, we will go through ways to test the API service endpoints in our recipe management application using Command Prompt. Testing is a very important step in application development. This is to ensure the functions we developed are working as expected. We can use curl or httpie, depending on your personal preference. In the subsequent exercise, we will show you both tools.

Curl (or cURL) is a command-line tool that can transfer data using URLs. We can use this tool to send requests to our API endpoints and examine the response. If you are running on macOS, you don't need to install curl. It is pre-installed in the system and you can find it in Terminal. You can also run it in the Terminal in PyCharm. However, if you are running on Windows, you need to download and install it for free from http://curl.haxx.se/download.html.

Httpie (aych-tee-tee-pie) is another command-line client that does a similar thing. It was built with the goal to improve the communication between the CLI (command-line interface) and the web. It is pretty user-friendly. For more details about httpie, please refer to https://httpie.org/.

We added httpie==1.0.2 in our requirements.txt previously, so PyCharm should have already installed it for us. The main benefit of having httpie is it will beautifully format the JSON document, making it more readable. And believe me, that will save us a lot of time when we move on to verifying the HTTP response from the server.

Exercise 3: Testing Our API Endpoints with httpie and curl

In this exercise, we are going to use httpie and curl to test our API endpoints. We will test the functions of getting all the recipes back from the server, and also creating/updating the recipes:

  1. We will first open the Terminal in PyCharm. It is located at the bottom of the application. It will look as shown in the following screenshot:
    Figure 1.9: PyCharm Terminal
    Figure 1.9: PyCharm Terminal
  2. Type in the following httpie command to get the recipes from our API endpoint, http://localhost:5000/recipes; we will be using the HTTP GET method here:
    http GET localhost:5000/recipes
  3. If you prefer to do it the curl way, use the following command instead. Note that we have different parameters here: -i is for showing the header in the response and -X is for specifying the HTTP method. We will be using GET here:
    curl -i -X GET localhost:5000/recipes 

    Note

    The http GET and curl-i -X GET commands basically do the same thing, which is using the HTTP GET method to send a request to http://localhost:5000/recipes. If the code that we put in on the server-side is working properly, the request will go through the /recipes route and the get_recipes function will be invoked. This will then get us all the recipes in JSON format.

    Take a look at the response we get. The first few lines in the response are the header. It has the HTTP status 200 OK and a Content-Length of 175 bytes. The Content-Type is application/json and, in the end, we have the response body in JSON format:

    HTTP/1.0 200 OK
    Content-Length: 175
    Content-Type: application/json
    Date: Mon, 15 Jul 2019 12:40:44 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "data": [
            {
                "description": "This is a lovely egg salad recipe.",
                "id": 1,
                "name": "Egg Salad"
            },
            {
                "description": "This is a lovely tomato pasta recipe.",
                "id": 2,
                "name": "Tomato Pasta"
            }
        ]
    }
  4. After that, let's create a recipe. This time, use the HTTP POST method, as we have lots of information that cannot be encoded in the URL. Please take a look at the following httpie command:
    http POST localhost:5000/recipes name="Cheese Pizza" description="This is a lovely cheese pizza"
  5. And then following is the curl command. The -H here is to specify the header in the request. Put in Content-Type: application/json, as we are going to send over the details of the new recipe in JSON format. The -d here is to specify the HTTP POST data, which is our new recipe:
    curl -i -X POST localhost:5000/recipes -H "Content-Type: application/json" -d '{"name":"Cheese Pizza", "description":"This is a lovely cheese pizza"}'
  6. The @app.route('/recipes', methods=['POST']) in the backend to catch this client request and invoke the create_recipe function. It will get the recipe details from the client request and save it to a list in the application memory. Once the recipe is successfully stored in the memory, it will return an HTTP status of 201 CREATED, and the new recipe will also be returned in the HTTP response for us to verify:
    HTTP/1.0 201 CREATED
    Content-Length: 77
    Content-Type: application/json
    Date: Mon, 15 Jul 2019 14:26:11 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "description": "This is a lovely cheese pizza",
        "id": 3,
        "name": "Cheese Pizza"
    }
  7. Now, get all the recipes again to verify if our previous recipe was really created successfully. We expect to receive three recipes in the response now:
    http GET localhost:5000/recipes 
    curl -i -X GET localhost:5000/recipes 
  8. Use either one of the preceding commands. They do the same thing, which is to trigger the get_recipes function and get us all the recipes currently stored in the application memory in JSON format.

    In the following response, we can see that the HTTP header is saying OK, and the Content-Length is now slightly longer than our previous response, that is, 252 bytes. This makes sense because we are expecting to see one more recipe in the response. The Content-Type is again application/json, with the body storing the recipes in JSON format. Now we can see our new recipe with ID 3:

    HTTP/1.0 200 OK
    Content-Length: 252
    Content-Type: application/json
    Date: Tue, 16 Jul 2019 01:55:30 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "data": [
            {
                "description": "This is a lovely egg salad recipe.",
                "id": 1,
                "name": "Egg Salad"
            },
            {
                "description": "This is a lovely tomato pasta recipe.",
                "id": 2,
                "name": "Tomato Pasta"
            },
            {
                "description": "This is a lovely cheese pizza",
                "id": 3,
                "name": "Cheese Pizza"
            }
        ]
    }
  9. Cool! So far, we are in pretty good shape. Now, test our application by trying to modify the recipe with ID 3. Use the HTTP PUT method and send over the modified name and description of the recipe to localhost:5000/recipes/3:
    http PUT localhost:5000/recipes/3 name="Lovely Cheese Pizza" description="This is a lovely cheese pizza recipe."

    The following is the curl command. Again, -H is to specify the header in the HTTP request, and we are setting that to "Content-Type: application/json"; -d is to specify that our data should be in JSON format:

    curl -i -X PUT localhost:5000/recipes/3 -H "Content-Type: application/json" -d '{"name":"Lovely Cheese Pizza", "description":"This is a lovely cheese pizza recipe."}'
  10. If things are working properly, then the client request will be caught by the @app.route('/recipes/<int:recipe_id>', methods=['PUT']) route. It will then invoke the update_recipe(recipe_id) function to look for the recipe with the passed-in recipe_id, update it, and return it. Together with the updated recipe in JSON format, we will also receive the HTTP status of OK (200):
    HTTP/1.0 200 OK
    Content-Length: 92
    Content-Type: application/json
    Date: Tue, 16 Jul 2019 02:04:57 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "description": "This is a lovely cheese pizza recipe.",
        "id": 3,
        "name": "Lovely Cheese Pizza"
    }
  11. Alright, all good so far. Now, go on and see if we can get a particular recipe. To do this, send a request to localhost:5000/recipes/3 to get the recipe with ID 3, and confirm whether our previous update was successful:
    http GET localhost:5000/recipes/3

    We can also use a curl command:

    curl -i -X GET localhost:5000/recipes/3 
  12. The application will look for the recipe with the recipe_id and return it in JSON format, together with an HTTP status of 200 OK:
    HTTP/1.0 200 OK
    Content-Length: 92
    Content-Type: application/json
    Date: Tue, 16 Jul 2019 06:10:49 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "description": "This is a lovely cheese pizza recipe.",
        "id": 3,
        "name": "Lovely Cheese Pizza"
    }
  13. Now, what if we try a recipe ID that we know doesn't exist? How will the application behave? Test it out with the httpie command as follows:
    http GET localhost:5000/recipes/101

    Alternatively, use the following curl command, which will do the same thing as in the preceding code:

    curl -i -X GET localhost:5000/recipes/101 
  14. Similarly, @app.route('/recipes/<int:recipe_id>', methods=['GET']) in the application will catch this client request and try to look for the recipe with ID = 101. The application will return with an HTTP status of 404 and a message: "recipe not found" in JSON format:
    HTTP/1.0 404 NOT FOUND
    Content-Length: 31
    Content-Type: application/json
    Date: Tue, 16 Jul 2019 06:15:31 GMT
    Server: Werkzeug/0.15.4 Python/3.7.0
    {
        "message": "recipe not found"
    }

If your application passed the test, congratulations! It is a pretty solid implementation. You can choose to perform more tests by yourself if you want to.

 

Postman

A Postman is a handy tool for API testing. It has a user-friendly GUI that we can send HTTP requests through. It allows us to send requests with different HTTP methods (that is, GET, POST, PUT, and DELETE) and we can check the response from the server. With this tool, we can easily test our API by sending a client request and checking the HTTP response. We can also save our test cases and group them into different collections.

The Postman GUI

We assume you should have already installed Postman by following the steps in the preface. When you open Postman, you should see the screen shown in the following screenshot. The left-hand side is a navigation panel for you to navigate through your historical or saved requests. In Postman, your requests are going to be organized into collections, which is like a folder in the filesystem. You can put relevant saved requests in the same collection.

The top panel is for you to compose your request. As you have learned from the command-line testing tool, we can have different HTTP verbs (such as GET and PUT). We also need to put in an API endpoint to send the request to. For some requests, you may also need to pass in additional parameters. These can all be done in Postman.

The bottom panel shows the server response:

Figure 1.10: Postman interface
Figure 1.10: Postman interface

Sending a GET Request

Sending a GET request is simple; we just need to fill in the target URL:

  1. Select GET as our HTTP method in the drop-down list.
  2. Enter the request URL (such as http://localhost:5000/API1).
  3. Click the Send button.

Sending a POST Request

Sending a POST request, however, will take a bit more work, because very often, we will put extra data in the request. For example, if you want to send some JSON data to an API endpoint, you can do the following:

  1. Select POST as our HTTP method in the drop-down list.
  2. Enter the request URL (such as http://localhost:5000/API2).
  3. Select the Body Tab. Also, select the "raw" radio button.
  4. Choose "JSON (application/json)" from the right drop-down menu. Put in the JSON data to the Body content area:
    {
         "key1": "value1",
         "key2": "value2"
    }
  5. Click the Send button.

Saving a Request

Very often, you will want to save your request for later use. This saving feature in Postman is particularly useful during regression testing. To save your request, you just need to click the save button, follow the on-screen instructions, and save it in a collection. Then you will see your saved request in the left navigation panel.

Note

You may need to open an account in Postman before you can save the request. Please follow the on-screen instructions accordingly.

If you want to learn more about Postman, click on the "Bootcamp" button at the bottom of Postman. You will see interactive tutorials showing you how to use Postman step-by-step on the screen.

Activity 1: Sending Requests to Our APIs Using Postman

Now that we have learned how to use Postman, we are going to test our application using Postman instead of the curl/httpie command-line testing tools. In this activity, we will be using this tool to test the CRUD functions in our web service:

  1. Create a request in Postman and get all the recipes.
  2. Use a POST request to create a recipe.
  3. Create a request to get all the recipes.
  4. Send an update request to modify the recipe that we have just created.
  5. Send a request to get a specific recipe.
  6. Send a request to search for a recipe that doesn't exist.

    Note

    The solution for this activity can be found on page 286.

If your application passed the test, congratulations! It is a pretty solid implementation.

Exercise 4: Automated Testing Using Postman

In this exercise, we would like to show you how we can use Postman as a powerful automatic testing tool. An automatic testing tool allows us to repeatedly send requests to the APIs, thus achieve testing automation. Postman allows us to do this. We can save historical requests in a collection so that you can reuse the same test cases next time:

  1. Hover the cursor over the request; the Save Request button will appear:
    Figure 1.11: Saving the request
    Figure 1.11: Saving the request
  2. Click on the Save Request button, and you will see a dialog box popping up, asking for more information. Type in Get all recipes for the request name and click on Create Collection at the bottom. Then, type in Basic API as the collection name and tick to confirm. Click Save to Basic API:
    Figure 1.12: Putting in information for saving the request
    Figure 1.12: Putting in information for saving the request
  3. The collection will then be created. Now, save our request to this collection for future use. We can also click on the Collections tab to see all the requests in that collection:
    Figure 1.13: Creating the new collection
Figure 1.13: Creating the new collection

Now we have a bunch of saved requests in our collection. Next time, if we make any changes in our application, we can rerun these tests to make sure the previously developed APIs are still working fine. This is called regression testing in the developer community. And Postman is a simple yet powerful tool for us to perform such testing.

Activity 2: Implement and Test the delete_recipe Function

Now we have a basic understanding of how to implement the API. We have coded the create and update recipe functions. In this activity, you will implement the delete_recipe function yourself.

You have learned about both the command-line and GUI testing tools. You will test the application using these tools after the implementation. This is what you need to do:

  1. Implement a delete_recipe function in app.py that can delete a specific recipe. Create the API endpoint accordingly.
  2. Start the application, make it ready for testing.
  3. Use httpie or curl to delete the recipe with ID = 1.
  4. Use Postman to delete the recipe with ID = 2.

    Note

    The solution for this activity can be found on page 291.

 

Summary

In this chapter, we have built a basic RESTful API using Flask. We did CRUD (Create, Read, Update, Delete) operations on our recipes, and through this, you should have grasped the concepts and fundamentals of APIs. We have also talked about relevant concepts, such as HTTP methods, HTTP status codes, JSON, and routing. We wrapped up the chapter by showing you different ways (command prompt, GUI) to test the web services that we have built.

After laying a good foundation, in the next chapter, we will continue to develop our recipe sharing platform step by step. You will learn the whole process of RESTful API development. Just stay with us, the best is yet to come!

About the Authors

  • Jack Chan

    Jack Chan started programming at the age of 10. He was an active participant in worldwide programming contests at university. Since graduation, he has been working in the finance and IT industries for more than 10 years, building systems that analyze millions of transactions and positions to spot suspicious activity. He has leveraged the powerful analytical Python libraries to perform data analysis and performance optimization for a trading system that works at a microsecond level. He has an in-depth knowledge of the modern software development life cycle, which uses automated testing, continuous integration, and agile methodologies. Among all programming languages, he found Python to be the most expressive and powerful. He has created courses and taught students all over the world, using Python as the teaching language. Inspiring aspiring developers to take on the software engineering career path has always been Jack's goal.

    Browse publications by this author
  • Ray Chung

    Ray Chung is a developer and an instructor. He loves helping students learn to code and master software development. He is now self-employed and develops web applications, network applications, and chatbots using Python. The first program he sold was a network application that helps clients configuring, maintaining, and testing thousands of multi-vendor network devices. He's experienced with big projects like a marathon's online registration system, rental car management system, and so on. He has worked extensively with Google App Engine, PostgreSQL, and advanced system architecture design. He has been a self-taught developer for many years and knows what is the most efficient way to learn a new skill.

    Browse publications by this author
  • Jack Huang

    Jack Huang is a programmer with an experience of more than 7 years in developing web applications in Python, JavaScript, and .NET. He is skilled in web frameworks such as Flask, Django, and Vue. He is also skilled in PostgreSQL, DynamoDB, MongoDB, RabbitMQ, Redis, Elasticsearch, RESTful APIs design, payment processing, system architecture design, database design, and UNIX systems. He has written applications for an accessories shop platform, the ERP system, divination web application, podcast platform, Job search service, blog system, salon reserving system, e-commerce service, and so on. He also has experience in handling large data and improving its performance and optimizing payment processing. He is an expert of web application developers who love coding and constantly following the newest technology.

    Browse publications by this author

Recommended For You

The Python Workshop

Cut through the noise and get real results with a step-by-step approach to learning Python 3.X programming

By Andrew Bird and 4 more
Python GUI Programming - A Complete Reference Guide

Explore Python’s GUI frameworks and create visually stunning and feature-rich applications

By Alan D. Moore and 1 more
Mastering PostgreSQL 12 - Third Edition

Master PostgreSQL 12 features such as advanced indexing, high availability, monitoring, and much more to efficiently manage and maintain your database

By Hans-Jürgen Schönig
Mastering Python Networking - Third Edition

New edition of the bestselling guide to mastering Python Networking, updated to Python 3 and including the latest on network data analysis, Cloud Networking, Ansible 2.8, and new libraries

By Eric Chou