Over the last few years, we have started taking for granted that data sources feeding for content, mobile device service feeds, or cloud computing are all driven by modern technologies, such as RESTful web services. Everybody has been talking about how their stateless model allows applications to scale easily and how it emphasizes on clear decoupling between data provisioning and data consumption. Nowadays, architects have started introducing the concept of microservices, aiming to reduce the complexity in systems by splitting their core components into small independent pieces that simply do a single task. So, enterprise-grade software is about to become a composite of such microservices. This makes it easy to maintain, and allows better life cycle management when new parts need to be introduced. Unsurprisingly, most of the microservices are serviced by RESTful frameworks. This fact may leave the impression that REST was invented sometime in the last decade, but that is far from the truth. In fact, REST has been around since the last decade of the previous century!
This chapter will walk you through the foundation of Representational State Transfer (REST) and will explain how REST couples with the HTTP protocol. You will look into five key principles that have to be considered while turning any HTTP application into a RESTful service-enabled application. You will also look at the differences between describing RESTful and classic Simple Object Access Protocol (SOAP)-based web services. Finally, you will learn how to utilize already-existing infrastructure for your benefit.
In this chapter, we will cover the following topics:
- REST fundamentals
- REST with HTTP
- Essential differences in the description, discovery, and documentation of RESTful services compared to classical SOAP-based services
- Taking advantage of existing infrastructure
It actually happened back in 1999, when a request for comments was submitted to the Internet Engineering Task Force (IETF; http://www.ietf.org/) via RFC 2616: Hypertext Transfer Protocol-HTTP/1.1. One of its authors, Roy Fielding, later defined a set of principles built around the HTTP and URI standards. This gave birth to REST as we know it today.
Note
These definitions were given in https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf in the fifth chapter, Representational State Transfer (REST), of Fielding's dissertation called Architectural Styles and the Design of Network-Based Software Architectures. The dissertation is still available at http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm.
Let's look at the key principles around the HTTP and URI standards, sticking to which will make your HTTP application a RESTful service-enabled application:
To understand this principle, one must conceive of the idea of representing data by a specific format and not by a physical file containing a bunch of bytes. Each piece of data available on the internet has a format that describes it, known as the content type; for example, JPEG images, MPEG videos, HTML, XML, text documents, and binary data are all resources with the following content types: image/jpeg, video/mpeg, text/html, text/xml, and application/octet-stream.
Since the internet contains so many different resources, they all should be accessible via URIs and should be identified uniquely. Furthermore, the URIs can be in a human-readable format, despite the fact that their consumers are more likely to be software programs rather than ordinary humans.
Human-readable URIs keep data self-descriptive and ease further development against it. This helps you to keep the risk of logical errors in your programs to a minimum.
Here are a few sample examples of such URIs representing different resources in a catalog application:
These human-readable URIs expose different types of resources in a straightforward manner. In the preceding example URIs, it is quite clear the data is items in a catalog, which are categorized watches. The first link shows all the items in the category. The second shows only the ones that are part of the 2018 collection. Next is a link pointing to the image of the item, followed by a link to a sample video. The last link points to a resource containing items from the previous collection in a ZIP archive. The media types served per each URI are rather easy to identify, with the assumption that the data format of an item is either JSON or XML, so we can easily map the media type of a self-described URL to one of the following:
- JSON or XML documents describing the items
- Images
- Videos
- Binary archive documents
The native HTTP protocol (RFC 2616) defines eight actions, also known as HTTP verbs:
GET
POST
PUT
DELETE
HEAD
OPTIONS
TRACE
CONNECT
The first four of them just feel natural in the context of resources, especially when defining actions for data manipulation. Let's make a parallel with relative SQL databases where the native language for data manipulation isCRUD(short forCreate, Read, Update, and Delete), originating from the different types of SQL statements, INSERT
,SELECT
,UPDATE
, andDELETE
, respectively. In the same manner, if you apply the REST principles correctly, the HTTP verbs should be used as shown here:
Note that a resource might be created by either the POST
or PUT
HTTP verbs, based on the policy of an application. However, if a resource has to be created under a specific URI with an identifier provided by the client, then PUT
is the appropriate action:
PUT /categories/watches/model-abc HTTP/1.1 Content-Type: text/xml Host: www.mycatalog.com <?xml version="1.0" encoding="utf-8"?> <Item category="watch"> <Brand>...</Brand> </Price></Price> </Item> HTTP/1.1 201 Created Content-Type: text/xml Location: http://www.mycatalog.com/categories/watches/model-abc
However, in your application, you may want to leave it up to the backend RESTful service to decide where to expose the newly created resource, and thus create it under an appropriate but still unknown or non-existent location.
For instance, in our example, we might want the server to define the identifier of newly created items. In such cases, just use the POST
verb to a URL without providing an identifier parameter. Then it is up to the service itself to provide a new unique and valid identifier for the new resource and to expose back this URL via the Location
header of the response:
POST /categories/watches HTTP/1.1 Content-Type: text/xml Host: www.mycatalog.com <?xml version="1.0" encoding="utf-8"?> <Item category="watch"> <Brand>...</Brand> </Price></Price> </Item> HTTP/1.1 201 Created Content-Type: text/xml Location: http://www.mycatalog.com/categories/watches/model-abc
A key feature of a resource is that it may be represented in a different format from the one in which it is stored. Thus, it can be requested or created in different representations. As long as the specified format is supported, the REST-enabled endpoint should use it. In the preceding example, we posted an XML representation of a watch item, but if the server had supported the JSON format, the following request would have been valid as well:
POST /categories/watches HTTP/1.1 Content-Type: application/json Host: www.mycatalog.com { "watch": { "id": ""watch-abc"", "brand": "...", "price": { "-currency": "EUR", "#text": "100" } } } HTTP/1.1 201 Created Content-Type: application/json Location: http://mycatalog.com/categories/watches/watch-abc
Resource manipulation operations through HTTP requests should always be considered atomic. All modifications of a resource should be carried out within an HTTP request in an isolated manner. After the request execution, the resource is left in a final state; this implicitly means that partial resource updates are not supported. You should always send the complete state of the resource.
Back to our catalog example, updating the price field of a given item would mean making a PUT request with a complete document (JSON or XML) that contains the entire data, including the updated price field. Posting only the updated price is not stateless, as it implies that the application is aware that the resource has a price field, that is, it knows its state.
Another requirement for your RESTful application to be stateless is that once the service gets deployed on a production environment, it is likely that incoming requests are served by a load balancer, ensuring scalability and high availability. Once exposed via a load balancer, the idea of keeping your application state at server side gets compromised. This doesn't mean that you are not allowed to keep the state of your application. It just means that you should keep it in a RESTful way. For example, keep a part of the state within the URI, or use HTTP headers to provide additional state-related data
The statelessness of your RESTful API isolates the caller against changes at the server side. Thus, the caller is not expected to communicate with the same server in consecutive requests. This allows easy application of changes within the server infrastructure, such as adding or removing nodes.
Note
Remember that it is your responsibility to keep your RESTful APIs stateless, as the consumers of the APIs would expect them to be.
Now that you know that REST is around 18 years old, a sensible question would be, "Why has it become so popular just quite recently?" Well, we the developers usually reject simple, straightforward approaches and, most of the time, prefer spending more time on turning already-complex solutions into even more complex and sophisticated ones.
Take classical SOAP web services, for example. Their various WS-* specifications are so many, and sometimes so loosely defined, that in order to make different solutions from different vendors interoperable, a separate specification, WS-Basic Profile, has been introduced. It defines extra interoperability rules in order to ensure that all WS-* specifications in SOAP-based web services can work together.
When it comes to transporting binary data with classical web services over HTTP, things get even more complex, as SOAP-based web services provide different ways of transporting binary data. Each way is defined in other sets of specifications, such as SOAP withAttachment References (SwaRef) and Message TransmissionOptimization Mechanism (MTOM). All this complexity was caused mainly because the initial idea of the web service was to execute business logic remotely, not to transport large amounts of data.
The real world has shown us that, when it comes to data transfer, things should not be that complex. This is where REST fits into the big picture—by introducing the concept of resources and a standard means for manipulating them.
Now that we've covered the main REST principles, it is time to dive deeper into what can be achieved when they are followed:
- Separation of the representation and the resource
- Visibility
- Reliability
- Scalability
- Performance
A resource is just a set of information, and as defined by principle 4, it can have multiple representations; however, its state is atomic. It is up to the caller to specify the desired media type with the Accept
header in the HTTP request, and then it is up to the server application to handle the representation accordingly, returning the appropriate content type of the resource together with a relevant HTTP status code:
HTTP 200 OK
in the case of successHTTP 400 Bad Request
if an unsupported format is given or for any other invalid request informationHTTP 406 Not Acceptable
if an unsupported media type is requested
HTTP 500 Internal Server Error
when something unexpected happens during the request processing
Let's assume that, at server side, we have items resources stored in an XML format. We can have an API that allows a consumer to request the item resources in various formats, such as application/xml
, application/json
, application/zip
, application/octet-stream
, and so on.
It would be up to the API itself to load the requested resource, transform it into the requested type (for example, JSON or XML), and either use ZIP to compress it or directly flush it to the HTTP response output.
The caller would make use of the Accept
HTTP header to specify the media type of the response they expect. So, if we want to request our item data inserted in the previous section in XML format, the following request should be executed:
GET /category/watches/watch-abc HTTP/1.1 Host: my-computer-hostname Accept: text/xml HTTP/1.1 200 OK Content-Type: text/xml <?xml version="1.0" encoding="utf-8"?> <Item category="watch"> <Brand>...</Brand> </Price></Price> </Item>
To request the same item in JSON format, the Accept
header needs to be set to application/json
:
GET /categoery/watches/watch-abc HTTP/1.1 Host: my-computer-hostname Accept: application/json HTTP/1.1 200 OK Content-Type: application/json { "watch": { "id": ""watch-abc"", "brand": "...", "price": { "-currency": "EUR", "#text": "100" } } }
REST is designed to be visible and simple. Visibility of the service means that every aspect of it should self-descriptive and follow the natural HTTP language according to principles 3, 4, and 5.
Visibility in the context of the outer world would mean that monitoring applications would be interested only in the HTTP communication between the REST service and the caller. Since the requests and responses are stateless and atomic, nothing more is needed to flow the behavior of the application and to understand whether anything has gone wrong.
Note
Remember that caching reduces the visibility of your RESTful applications and in general should be avoided, unless needed for serving resources subject to large amounts of callers. In such cases, caching may be an option, after carefully evaluating the possible consequences of serving obsolete data.
Before talking about reliability, we need to define which HTTP methods are safe and which are idempotent in the REST context. So, let's first define what safe and idempotent methods are:
- An HTTP method is considered to be safe provided that, when requested, it does not modify or cause any side effects on the state of the resource
- An HTTP method is considered to be idempotent if its response stays the same, regardless of the number of times it is requested, am idempotent request always gives back the same request, if repeated identically.
The following table lists which HTTP methods are safe and which are idempotent:
HTTP method | Safe | Idempotent |
| Yes | Yes |
| No | No |
| No | Yes |
| No | Yes |
Consumers should consider operation's safety and the idempotence features in order to be served reliably.
So far, we stressed the importance of having stateless behavior for a RESTful web application. The World Wide Web (WWW) is an enormous universe, containing huge amount of data and a lot of users, eager to get that data. The evolution of the WWW has brought the requirement that applications should scale easily as their load increases. Scaling applications that have a state is difficult to achieve, especially when zero or close-to-zero operational downtime is expected.
That's why staying stateless is crucial for any application that needs to scale. In the best-case scenario, scaling your application would require you to put another piece of hardware for a load balancer, or bring another instance in your cloud environment. There would be no need for the different nodes to sync between each other, as they should not care about the state at all. Scalability is all about serving all your clients in an acceptable amount of time. Its main idea is to keep your application running and to prevent Denial of Service (DoS) caused by a huge amount of incoming requests.
Scalability should not be confused with the performance of an application. Performance is measured by the time needed for a single request to be processed, not by the total number of requests that the application can handle. The asynchronous non-blocking architecture and event-driven design of Node.js make it a logical choice for implementing an application that scales and performs well.
If you are familiar with SOAP web services, you may have heard of the Web Service Definition Language (WSDL). It is an XML description of the interface of the service and defines an endpoint URL for invocation. It is mandatory for a SOAP web service to be described by such a WSDL definition.
Similar to SOAP web services, RESTful services can also make use of a description language, called WADL. WADL stands for Web Application Definition Language. Unlike WSDL for SOAP web services, a WADL description of a RESTful service is optional, that is, consuming the service has nothing to do with its description.
Here is a sample part of a WADL file that describes the GET
operation of our catalog service:
<?xml version="1.0" encoding="UTF-8"?><applicationxmlns="http://wadl.dev.java.net/2009/02"xmlns:service="http://localhost:8080/catalog/"xmlns:xsd="http://www.w3.org/2001/XMLSchema"><grammer><includehref="items.xsd" /><includehref="error.xsd" /></grammer><resourcesbase="http://localhost:8080/catalog/categories"><resourcepath="{category}"><methodname="GET"><request><paramname="category"type="xsd:string"style="template" /></request><responsestatus="200"><representationmediaType="application/xml"element="service:item" /><representationmediaType="application/json" /></response><responsestatus="404"><representationmediaType="application/xml"element="service:item" /></response></method></resource></resources></application>
This extract of a WADL file shows how application, exposing resources is described. Briefly, each resource must be part of an application. The resource provides a, where it is located with the base
attribute, and describes each of its supported HTTP methods in a method. Additionally, an optional doc
element can be used at resource and application to provide additional documentation about the service and its operations.
Though WADL is optional, it significantly reduces the efforts of discovering RESTful services.
Public APIs exposed on the web should be well documented, otherwise it would be difficult for developers to use them in their applications. While WADL definitions might be considered a source of documentation, they address a different problem—the discovery of the service. They serve metadata for the services to machines, not to humans. The Swagger project (https://swagger.io/) addresses the need for neat documentation of RESTful APIs. It defines a meta description of an API from an almost human-readable JSON format. The following is a sample swagger.json
file, partially describing the catalog service:
{ "swagger": "2.0", "info": { "title": "Catalog API Documentation", "version": "v1" }, "paths": { "/categories/{id}" : { "get": { "operationId": "getCategoryV1", "summary": "Get a specific category ", "produces": [ "application/json" ], "responses": { "200": { "description": "200 OK", "examples": {"application/json": { "id": 1, "name": "Watches", "itemsCount": 550 } } }, "404": {"description" : "404 Not Found"}, "500": {"description": "500 Internal Server Error"} } } } }, "consumes": ["application/json"] }
The swagger.json
file is really straightforward: it defines a name and version of your API and gives a brief description of each operation it exposes, nicely coupled with a sample payload. But the real benefit from it comes in another subproject of Swagger, called swagger-ui
(https://swagger.io/swagger-ui/), which actually renders this data from swagger.json
nicely into an interactive web page that only provides documentation, but also allows interaction with the service:
We will have a look at and utilize the swagger-ui
Node.js module to provide the API that we will develop later in the book, with up-to-date documentation.
The best part of developing and distributing RESTful applications is that the infrastructure needed is already out there, available to you. As RESTful applications use the existing web space heavily, you need to do nothing more than follow the REST principles when developing. In addition, there are plenty of libraries available out there for any platform, and I do mean any platform. This eases the development of RESTful applications, so you just need to choose your preferred platform and start developing.
In this chapter, you learned about foundation of a REST, looking at five key principles that transform a web application into a REST-enabled application. We made a brief comparison between RESTful services and classical SOAP web services, and finally took a look at how RESTful services are documented and how we can simplify the discovery of the services we develop.
Now that you know the basics, we are ready to dive into the Node.js way of implementing RESTful services. In the next chapter, you will learn about the essentials of Node.js and the accompanying tools that it is necessary to use and understand in order to build a real-life fully-fledged web service.