Hands-On RESTful Web Services with Go - Second Edition

By Naren Yellavula
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Getting Started with REST API Development

About this book

Building RESTful web services can be tough as there are countless standards and ways to develop API. In modern architectures such as microservices, RESTful APIs are common in communication, making idiomatic and scalable API development crucial. This book covers basic through to advanced API development concepts and supporting tools.

You’ll start with an introduction to REST API development before moving on to building the essential blocks for working with Go. You’ll explore routers, middleware, and available open source web development solutions in Go to create robust APIs, and understand the application and database layers to build RESTful web services. You’ll learn various data formats like protocol buffers and JSON, and understand how to serve them over HTTP and gRPC. After covering advanced topics such as asynchronous API design and GraphQL for building scalable web services, you’ll discover how microservices can benefit from REST. You’ll also explore packaging artifacts in the form of containers and understand how to set up an ideal deployment ecosystem for web services. Finally, you’ll cover the provisioning of infrastructure using infrastructure as code (IaC) and secure your REST API.

By the end of the book, you’ll have intermediate knowledge of web service development and be able to apply the skills you’ve learned in a practical way.

Publication date:
February 2020
Publisher
Packt
Pages
404
ISBN
9781838643577

 

Getting Started with REST API Development

A web service is a communication mechanism defined between various computer systems. Without web services, custom peer-to-peer communication becomes cumbersome and platform-specific. The web needs to understand and interpret a hundred different things in the form of protocols. If computer systems can align with the protocols that the web can understand easily, it is a great help.

A web service is a software system designed to support interoperable machine-to-machine interaction over a network, as defined by the World Wide Web Consortium (W3C) at https://www.w3.org/TR/ws-arch/.

Now, in simple words, a web service is a road between two endpoints where messages are transferred smoothly. The message transfer is usually one way. Two individual programmable entities can also communicate with each other through their own APIs. Two people communicate through language, two applications communicate through an Application Programming Interface (API).

The reader might be wondering; what is the importance of the API in the current digital world? The rise of the Internet of Things (IoT) made API usage heavier than before. Awareness of APIs is growing day by day, and there are hundreds of APIs that are being developed and documented all over the world every day. Notable major businesses are seeing the future in the API as a Service (AaS). A bright example in recent times is Amazon Web Services (AWS). AWS is a huge success in the cloud world. Developers write their own applications using the Representational State Transfer (REST) API provided by AWS and access it via Command-Line Interface (CLI). 

A few more hidden use cases are from travel sites such as http://Booking.com and https://www.expedia.com/, which fetch real-time prices by calling the APIs of third-party gateways and data vendors. Web service usage is often charged these days by the amount of data requests.

In this chapter, we will focus on the following topics:

  • The different web services available
  • REST architecture in detail
  • The rise of Single-page applications (SPAs) with REST
  • Setting up a Go project and running a development server
  • Building our first service for finding the fastest mirror site from a list of Debian servers hosted worldwide
  • The Open API specification and Swagger documentation
 

Technical requirements

The following are the pieces of software that should be pre-installed for running the code samples in this chapter:

  • OS: Linux (Ubuntu 18.04)/ Windows 10/Mac OS X >=10.13
  • Software: Docker >= 18 (Docker Desktop for Windows and Mac OS X)
  • The latest version of the Go compiler == 1.13.5

We use Docker in this book to run a few sandbox environments. Docker is a virtualization platform that imitates an OS in a sandbox. Using it, we can cleanly run an application or service without affecting the host system.

You can find the code used in this chapter on the book's GitHub repository at https://github.com/PacktPublishing/Hands-On-Restful-Web-services-with-Go/tree/master/chapter1.

 

Types of web services

There are many types of web services that have evolved over time. Some of the more prominent ones are as follows:

  • Simple Object Access Protocol (SOAP)
  • Universal Description, Discovery, and Integration (UDDI)
  • Web Services Description Language (WSDL)
  • Representational State Transfer (REST)

Out of these, SOAP became popular in the early 2000s, when XML riding on a high wave. The XML data format is used by various distributed systems to communicate with each other.

A SOAP request usually consists of these three basic components:

  • The envelope
  • The header
  • The body

Just to perform an HTTP request and response cycle, we have to attach a lot of additional data in SOAP. A sample SOAP request to a fictional book server, www.example.org, looks like this:

POST /Books HTTP/1.1
Host: www.example.org
Content-Type: application/soap+xml; charset=utf-8
Content-Length: 299
SOAPAction: "https://www.w3.org/2003/05/soap-envelope"

<?xml version="1.0"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:m="https://www.example.org">
<soap:Header>
</soap:Header>
<soap:Body>
<m:GetBook>
<m:BookName>Alice in the wonderland</m:BookName>
</m:GetBook>
</soap:Body>
</soap:Envelope>

This is a standard example of a SOAP request for getting book data. If we observe carefully, it is in XML format, with special tags specifying the envelope and body. Since XML works by defining a lot of namespaces, the response gets bulky.

The main drawback of SOAP is that it is too complex for implementing web services and is a heavyweight framework. A SOAP HTTP request can get very bulky and can cause bandwidth wastage. The experts looked for a simple alternative, and in came REST. In the next section, we will briefly discuss REST.

The REST API

The name Representational state transfer (RESTwas coined by Roy Fielding from the University of California. It is a very simplified and lightweight web service compared to SOAP. Performance, scalability, simplicity, portability, and flexibility are the main principles behind the REST design. 

The REST API allows different systems to communicate and send/receive data in a very simple way. Each and every REST API call has a relation between an HTTP verb and the URL. The resources in the database in an application can be mapped with an API endpoint in the REST architecture.

When you are using a mobile app on your phone, your phone might be talking to many cloud services to retrieve, update, or delete your data. REST services have a huge impact on our daily lives.

REST is a stateless, cacheable, and simple architecture that is not a protocol, but a pattern. This pattern allows different endpoints to communicate with each other over HTTP.

Characteristics of REST services

These are the main properties that make REST simple and unique compared to its predecessors:

  • Client-server based architecture: This architecture is most essential for the modern web to communicate over HTTP. A single client-server may look naive initially, but many hybrid architectures are evolving. We will discuss more of these shortly.
  • Stateless: This is the most important characteristic of a REST service. A REST HTTP request consists of all the data needed by the server to understand and return the response. Once a request is served, the server doesn't remember whether the request arrived after a while. So, the operation will be a stateless one.
  • Cacheable: In order to scale an application well, we need to cache certain responses. REST services can be cached for better throughput.
  • Representation of resources: The REST API provides the uniform interface to talk to. It uses a Uniform Resource Identifier (URI) to map the resources (data). It also has the advantage of requesting a specific data
  • Implementation freedom: REST is just a mechanism to define your web services. It is an architectural style that can be implemented in multiple ways. Because of this flexibility, you can create REST services in the way you wish to. As long as it follows the principles of REST, you have the freedom to choose the platform or technology for your server. 
 Thoughtful caching is essential for REST services to scale.

We have seen the types of web services and understood what is REST API. We also looked at the characteristics that make REST services unique. In the next section, we will take a look at REST verbs and status code and cover a few examples of path parameters. 

 

REST verbs and status codes

REST verbs specify an action to be performed on a specific resource or a collection of resources. When a request is made by the client, it should send the following information in the HTTP request:

  • The REST verb
  • Header information
  • The body (optional)

As we mentioned previously, REST uses the URI to decode the resource to be handled. There are quite a few REST verbs available, but six of them are used particularly frequently. They are presented, along with their expected actions, in the following table:

REST Verb Action
GET Fetches a record or set of resources from the server
OPTIONS Fetches all available REST operations 
POST Creates a resource or a new set of resources
PUT Updates or replaces the given record
PATCH Modifies the given record
DELETE Deletes the given resource 

 

The status of these operations can be known from HTTP status codes. Whenever a client initiates a REST operation, since REST is stateless, the client should know a way to find out whether the operation was successful or not. For that reason, HTTP responses have a status code. REST defines a few standard status code types for a given operation. This means a REST API should strictly follow the following rules to achieve stable results in client-server communication. There are three important ranges available based on the types of error. See the following table for error ranges:

Status Code Type Number Range Action
Success 200 - 226  The 2xx family is used for successful responses.
Error 400 - 499 (client), 500 - 599 (server) The 4xx family is used for indicating client errors. The 5xx is for server failures to process the request.
Redirect 300 - 308 The 3xx family is for URL redirection.

 

The detail of what each status code does is very precisely defined, and the overall count of codes increases every year. We mention the important ones in the upcoming section.

All requests to REST services have the following format. It consists of the host and the API endpoint. The API endpoint is the URL path that is predefined by the server. It can also include optional query parameters.

A trivial REST API URI looks like the following: http://HostName/APIEndpoint/?key=value(optional)

Let's look at all the verbs in more detail. The REST API design starts with the defining of operations and API endpoints. Before implementing the API, the design document should list all the endpoints for the given resources.

In the following section, we carefully observe the REST API endpoints using PayPal's REST API as a use case.

GET

A GET method fetches the given resource from the server. To specify a resource, GET uses a few types of URI queries:

  • Query parameters
  • Path-based parameters

In case you didn't know, most of your browsing of the web is done by performing a GET request to the server. For example, if you type www.google.com, you are actually making a GET request to fetch the search page. Here, your browser is the client and Google's web server is the backend implementer of web services. A successful GET operation returns a 200 status code.

Examples of path parameters

Everyone knows PayPal. PayPal creates billing agreements with companies. If you register with PayPal for a payment system, they provide you with a REST API for all your billing needs. The sample GET request for getting the information of a billing agreement looks like this: /v1/payments/billing-agreements/agreement_id.

Here, the resource query is with the path parameter. When the server sees this line, it interprets it as I got an HTTP request with a need for agreement_id from the billing agreements. Then it searches through the database, goes to the billing-agreements table, and finds an agreement with the given agreement_id. If that resource exists, it sends back a copy of the details in response (200 OK), or else it sends a response saying, "resource not found" (404).

Using GET, you can also query a list of resources, instead of a single one as in the preceding example. PayPal's API for getting billing transactions related to an agreement can be fetched with /v1/payments/billing-agreements/transactions. This line fetches all transactions that occurred on that billing agreement. In both instances, the data is retrieved in the form of a JSON response. The response format should be designed beforehand so that the client can consume it in the agreement.

Examples of query parameters are as follows:

  • Query parameters are intended to add detailed information to identify a resource from the server. For example, imagine a sample fictitious API. Let's assume this API is created for fetching, creating, and updating the details of the book. A query parameter based GET request will be in this format: 
/v1/books/?category=fiction&publish_date=2017
  • The preceding URI has a couple of query parameters. The URI is requesting a book from the books resource that satisfies the following conditions:
  • It should be a fiction book
  • The book should have been published in the year 2017

Get all the fiction books that are released in the year 2017 is the question the client is posing to the server. 

Path vs Query parametersWhen to use them? It is a common rule of thumb that Query parameters are used to fetch multiple resources based on the Query parameters. If a client needs a single resource with exact URI information, it can use Path parameters to specify the resource. For example, a user dashboard can be requested with Path parameters, and fetch data on filtering can be modeled with Query parameters.

Use Path parameters for a single resource and Query parameters for multiple resources in a GET request.

POST, PUT, and PATCH 

The POST method is used to create a resource on the server. In the previous books API, this operation creates a new book with the given details. A successful POST operation returns a 2xx status code. The POST request can update multiple resources: /v1/books.

The POST request can have a JSON body like the following: 

{"name" : "Lord of the rings", "year": 1954, "author" : "J. R. R. Tolkien"}

This actually creates a new book in the database. An ID is assigned to this record so that when we GET the resource, the URL is created. So, POST should be done only once, in the beginning. In fact, Lord of the Rings was published in 1955. So, we entered the published date incorrectly. In order to update the resource, let's use the PUT request.

The PUT method is similar to POST. It is used to replace the resource that already exists. The main difference is that PUT is an idempotent operation. A POST call creates two instances with the same data. But PUT updates a single resource that already exists:

/v1/books/1256

PUT does this using a body containing JSON syntax, as follows:

{"name" : "Lord of the rings", "year": 1955, "author" : "J. R. R. Tolkien"}

1256 is the ID of the book. It updates the preceding book with year:1955. Did you observe the drawback of PUT? It actually replaced the entire old record with the new one. We needed to change a single column. But PUT replaced the whole record. That is bad. For this reason, the PATCH request was introduced. 

The PATCH method is similar to PUT, except it won't replace the whole record. PATCH, as the name suggests, patches the column that is being modified. Let's update the book 1256 with a new column called ISBN:

/v1/books/1256

Let's use put the following JSON in the body:

{"isbn" : "0618640150"}

It tells the server, "search for the book with ID 1256. Then add/modify this column with the given value."

 PUT and PATCH both return a 2xx status for success and 404 for not found.

DELETE and OPTIONS

The DELETE API method is used to delete a resource from the database. It is similar to PUT but without a body. It just needs an ID of the resource to be deleted. Once a resource gets deleted, subsequent GET requests return a 404 not found status. 

Responses to this method are not cacheable (should caching be implemented) because the DELETE method is idempotent.

The OPTIONS API method is the most underrated in API development. Given the resource, this method tries to find all possible methods (GET, POST, and so on) defined on the server. It is like looking at the menu card at a restaurant and then ordering an item that is available (whereas if you randomly order a dish, the waiter will tell you it is not available). It is best practice to implement the OPTIONS method on the server. From the client, make sure OPTIONS is called first, and if the method is available, then proceed with it. 

Cross-Origin Resource Sharing (CORS)

The most important application of this OPTIONS method is Cross-Origin Resource Sharing (CORS). Initially, browser security prevented the client from making cross-origin requests. It means a site loaded with the www.foo.com URL can only make API calls to that host. If the client code needs to request files or data from www.bar.com, then the second server, bar.com, should have a mechanism to recognize foo.com to get its resources.

The following is the diagram depicting the CORS process:

Let's examine the steps followed in the preceding CORS diagram:

  1. foo.com requests the OPTIONS method on bar.com
  2. bar.com sends a header like Access-Control-Allow-Origin: http://foo.com in response to the client
  3. Next, foo.com can access the resources on bar.com without any restrictions that call any REST method

If bar.com feels like supplying resources to any host after one initial request, it can set the access control to *.

In the next section, we see why the REST API plays such a major role in the next generation of web services. SPAs made it possible to leverage APIs for all purposes, including the UI, clients, and so on.

 

The rise of the REST API with SPAs

Let's try to understand why SPAs are already standards of today's web. Instead of building a UI in the traditional way (that is, requesting rendered web pages), SPA designs allow developers to write code in a totally different way. There are many Model-View-Controller (MVC) frameworks, including Angular, React, Vue.js, and so on, for developing web UIs rapidly, but the essence of each of them is pretty simple. All MVC frameworks help us to implement one design pattern. That design pattern is no requesting of web pages, only REST API usage.

Modern frontend web development has advanced a lot in the last decade (2010-2020). In order to exploit the features of the MVC architecture, we have to consider the frontend as a separate entity that talks to the backend only using the REST API (preferably using JSON data).

Old and new methods of data flow in SPA

In the traditional flow of serving requests, the order looks like this:

  1. The client requests a web page from the server
  2. The server authenticates and returns a rendered response
  3. Every rendered response is in HTML with embedded data

With SPAs, however, the flow is quite different:

  1. Request the HTML templates with the browser in one single go
  2. Then, query the JSON REST API to fill a model (the data object)
  3. Adjust the UI according to the data in the model (in JSON)
  4. From the browser, push back the changes to the server via an API call

In this way, communication happens only in the form of the REST API. The client takes care of logically representing the data. This causes systems to move from Response-Oriented Architecture (ROA) to Service-Oriented Architecture (SOA). Take a look at the following diagram:

SPAs reduce bandwidth usage and improve site performance. SPAs are a major boost for API-centric server development because now a server can satisfy requirements for both browser and API clients.

Why use Go for REST API development?

REST services are trivial in the modern web. SOA (which we discuss in more detail later) created an activity space for REST services to take web development to the next level. Go is a programming language from the house of Google for solving the bigger problems they have. It has been over ten years since its first appearance. It matured along the way with the developer community jumping in and creating huge-scale systems in it.

Go is the darling of the web. It solves bigger problems in an easy way.

We could choose Python or JavaScript (Node.js) for our REST API development, but the main advantage of Go lies in its speed and compile-time error detection. Go has been proven to be faster than dynamic programming languages in terms of computational performance according to various benchmarks. These are the three reasons why a company should write their next API in Go:

  • To scale your API for a wider audience
  • To enable your developers to build robust systems
  • To start simple and go big

As we progress through this book, we learn how to build efficient REST services in Go.

 

Setting up the project and running the development server

This is a building series book. It assumes you already know the basics of Go. If not, no worries. You can get a jump-start and learn the basics quickly from Go's official site at https://golang.org/. Writing a simple standalone program with Go is straightforward. But for big projects, we have to set up a clean project layout. For that reason, as a Go developer, you should know how Go projects are laid out and the best practices to keep your code clean.

Make sure you have done the following things before proceeding:

  • Install the Go compiler on your machine
  • Set the GOROOT and GOPATH environment variables

There are many online references from which you can get to know the preceding details. Depending on your machine type (Windows, Linux, or Mac OS X ), set up a working Go compiler. We will see more details about GOPATH in the following section.

Demystifying GOPATH 

GOPATH is nothing but the current appointed workspace on your machine. It is an environment variable that tells the Go compiler where your source code, binaries, and packages are placed.

The programmers coming from a Python background may be familiar with the Virtualenv tool for creating multiple projects (with different Python interpreter versions) at the same time. But at a given time, you can activate the environment for the project that you wish to work on and develop your project. Similarly, you can have any number of Go projects on your machine. While developing, set the GOPATH to one of your projects. The Go compiler now activates that project.

It is a common practice to create a project under the home directory and set the GOPATH environment variable as follows:

mkdir /home/user/workspace
export GOPATH=/home/user/workspace

Now, we install external packages like this:

go get -u -v github.com/gorilla/mux

 Go copies a project called mux from GitHub into the currently activated project workspace.

For go get, use the -u flag to install the updated dependencies of the external package, and -v to see the verbose details of the installation.

A typical Go project, hello, should reside in the src directory in GOPATH, as mentioned on the official Go website:

  

Let's understand this structure before digging further:

  • bin: Stores the binary of our project; a shippable binary that can be run directly
  • pkg: Contains the package objects; a compiled program that supplies package methods
  • src: The place for your project source code, tests, and user packages

In Go, all the packages imported into the main program have an identical structure, github.com/user/project. But who creates all these directories? Should the developer do that? Yes. It is the developer's responsibility to create directories for their project. It means they only create the src/github.com/user/hello directory. 

When a developer runs the install command, the bin and package directories are created if they did not exist before. .bin consists of the binary of our project source code and .pkg consists of all internal and external packages we use in our Go programs:

 go install github.com/user/project

Let's build a small service to brush up on our Go language skills. Operating systems such as Debian and Ubuntu host their release images on multiple FTP servers. These are called mirrors. Mirrors are helpful in serving an OS image from the closest point to a client. Let's build a service that finds the fastest mirror from a list of mirrors.

 

Building our first service – finding the fastest mirror site from a list

With the concepts we have built up to now, let's write our first REST service. Many mirror sites exist for hosting operating system images including Ubuntu and Debian. The mirror sites here are nothing but websites on which OS images are hosted to be geographically close to the downloading machines.

Let's look at how we can create our first service:

Problem:

Build a REST service that returns the information of the fastest mirror to download a given OS from a huge list of mirrors. Let's take the Debian OS mirror list for this service. You can find the list at https://www.debian.org/mirror/list.

We use that list as input when implementing our service.

Design:

Our REST API should return the URL of the fastest mirror.

The block of the API design document may look like this:

HTTP Verb PATH Action Resource
GET /fastest-mirror fetch URL: string

 

Implementation:

Now we are going to implement the preceding API step by step:

The code for this project is available at https://github.com/PacktPublishing/Hands-On-Restful-Web-services-with-Go in the chapter1 subdirectory.
  1. As we previously discussed, you should set the GOPATH variable first. Let's assume the GOPATH variable is /home/user/workspace. Create a directory called mirrorFinder in the following path. git-user should be replaced with your GitHub username under which this project resides:
mkdir -p $GOPATH/src/github.com/git-user/chapter1/mirrorFinder
  1. Our project is ready. We don't have any data store configured yet. Create an empty file called main.go:
touch $GOPATH/src/github.com/git-user/chapter1/mirrorFinder/main.go
  1. Our main logic for the API server goes into this file. For now, we can create a data file that works as a data service for our main program. Create one more directory for packaging the mirror list data: 
mkdir $GOPATH/src/github.com/git-user/chapter1/mirrors
  1. Now, create an empty file called data.go in the mirrors directory. The src directory structure so far looks like this:
github.com \
-- git-user \
-- chapter1
-- mirrorFinder \
-- main.go
-- mirrors \
-- data.go
  1. Let's start adding code to the files. Create an input data file called data.go for our API to use:
package mirrors

// MirrorList is list of Debian mirror sites
var MirrorList = [...]string{
"http://ftp.am.debian.org/debian/", "http://ftp.au.debian.org/debian/",
"http://ftp.at.debian.org/debian/", "http://ftp.by.debian.org/debian/",
"http://ftp.be.debian.org/debian/", "http://ftp.br.debian.org/debian/",
"http://ftp.bg.debian.org/debian/", "http://ftp.ca.debian.org/debian/",
"http://ftp.cl.debian.org/debian/", "http://ftp2.cn.debian.org/debian/",
"http://ftp.cn.debian.org/debian/", "http://ftp.hr.debian.org/debian/",
"http://ftp.cz.debian.org/debian/", "http://ftp.dk.debian.org/debian/",
"http://ftp.sv.debian.org/debian/", "http://ftp.ee.debian.org/debian/",
"http://ftp.fr.debian.org/debian/", "http://ftp2.de.debian.org/debian/",
"http://ftp.de.debian.org/debian/", "http://ftp.gr.debian.org/debian/",
"http://ftp.hk.debian.org/debian/", "http://ftp.hu.debian.org/debian/",
"http://ftp.is.debian.org/debian/", "http://ftp.it.debian.org/debian/",
"http://ftp.jp.debian.org/debian/", "http://ftp.kr.debian.org/debian/",
"http://ftp.lt.debian.org/debian/", "http://ftp.mx.debian.org/debian/",
"http://ftp.md.debian.org/debian/", "http://ftp.nl.debian.org/debian/",
"http://ftp.nc.debian.org/debian/", "http://ftp.nz.debian.org/debian/",
"http://ftp.no.debian.org/debian/", "http://ftp.pl.debian.org/debian/",
"http://ftp.pt.debian.org/debian/", "http://ftp.ro.debian.org/debian/",
"http://ftp.ru.debian.org/debian/", "http://ftp.sg.debian.org/debian/",
"http://ftp.sk.debian.org/debian/", "http://ftp.si.debian.org/debian/",
"http://ftp.es.debian.org/debian/", "http://ftp.fi.debian.org/debian/",
"http://ftp.se.debian.org/debian/", "http://ftp.ch.debian.org/debian/",
"http://ftp.tw.debian.org/debian/", "http://ftp.tr.debian.org/debian/",
"http://ftp.uk.debian.org/debian/", "http://ftp.us.debian.org/debian/",
}

We create a map of strings called MirrorList. This map holds information on the URL to reach the mirror site. We are going to import this information into our main program to serve the request from the client.

  1. Open main.go and add the following code:
import (
"encoding/json"
"fmt"
"log"
"net/http"
"time"

"github.com/git-user/chapter1/mirrors"
)


type response struct {
FastestURL string `json:"fastest_url"`
Latency time.Duration `json:"latency"`
}


func main() {
http.HandleFunc("/fastest-mirror", func(w http.ResponseWriter,
r *http.Request) {
response := findFastest(mirrors.MirrorList)
respJSON, _ := json.Marshal(response)
w.Header().Set("Content-Type", "application/json")
w.Write(respJSON)
})
port := ":8000"
server := &http.Server{
Addr: port,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
fmt.Printf("Starting server on port %sn", port)
log.Fatal(server.ListenAndServe())
}

We created the main function that runs an HTTP server. Go provides the net/http package for that purpose. The response of our API is a struct with two fields:

  • fastest_url: The fastest mirror site
  • latency: The time it takes to download the README from the Debian OS repository
  1. We will code a function called findFastest to make requests to all the mirrors and calculate the fastest of all. To do this, instead of making sequential API calls to each and every URL one after the other, we use Go routines to parallelly request the URLs and once a goroutine returns, we stop there and return that data back.:
func findFastest(urls []string) response {
urlChan := make(chan string)
latencyChan := make(chan time.Duration)

for _, url := range urls {
mirrorURL := url
go func() {
start := time.Now()
_, err := http.Get(mirrorURL + "/README")
latency := time.Now().Sub(start) / time.Millisecond
if err == nil {
urlChan <- mirrorURL
latencyChan <- latency
}
}()
}
return response{<-urlChan, <-latencyChan}
}

The findFastest function is taking a list of URLs and returning the response struct. The function creates a goroutine per mirror site URL. It also creates two channels, urlChan and latencyChan, which are passed to the goroutines. In the goroutines, we calculate the latency (time taken for the request).

The smart logic here is, whenever a goroutine receives a response, it writes data into two channels with the URL and latency information respectively. Upon receiving data, the two channels make the response struct and return from the findFastest function. When that function is returned, all goroutines spawned from that are stopped from whatever they are doing. So, we will have the shortest URL in urlChan and the smallest latency in latencyChan.

  1. Now if you add this function to the main file (main.go), our code is complete for the task:
Always use the Go fmt tool to format your Go code. Some example usage of fmt looks like the following: go fmt github.com/narenaryan/romanserver
  1. Now, install this project with the Go command, install:
go install github.com/git-user/chapter1/mirrorFinder

This step does two things:

  • Compiles the package mirrors and places a copy in the $GOPATH/pkg directory
  • Places a binary in the $GOPATH/bin
  1. We can run the preceding API server like this:
$GOPATH/bin/mirrorFinder

The server is up and running on http://localhost:8000. Now we can make a GET request to the API using a client such as a browser or a curl command. Let's fire a curl command with a proper API GET request.

Request one is as follows:

curl -i -X GET "http://localhost:8000/fastest-mirror" # Valid request

The response is as follows:

HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 27 Mar 2019 23:13:42 GMT
Content-Length: 64

{"fastest_url":"http://ftp.sk.debian.org/debian/","latency":230}

Our fastest-mirror-finding API is working great. The right status code is being returned. The output may change with each API call, but it fetches the lowest-latency link at any given moment. This example also shows where goroutines and channels shine.

In the next section, we'll look at an API specification called Open API. An API specification is for documenting the REST API. To visualize the specification, we will use the Swagger UI tool.

 

Open API and Swagger

Because APIs are very common, the Open API Specification is a community-driven open specification within the OpenAPI Initiative, a Linux Foundation Collaborative Project.

The OpenAPI Specification (OAS), formerly called the Swagger Specification, is an API description format for REST APIs. An Open API file allows you to describe your entire API, including the following:

  • Available endpoints 
  • Endpoint operations (GET, PUT, DELETE, and so on)
  • Parameter input and output for each operation
  • Authentication methods
  • Contact information, license, terms of use, and other information.

Open API has many versions and is rapidly developing. The current stable version is 3.0.

There are two formats, JSON and YAML, that are supported by OAS. Swagger and Open API both are different. Swagger has many products, including the following:

  • Swagger UI (for validating Open API files and interactive docs)
  • Swagger Codegen (for generating server stubs)

Whenever we develop a REST API, it is a better practice to create an Open API/Swagger file that captures all the necessary details and descriptions of the API. The file can then be used in Swagger UI to create interactive documentation.

Installing Swagger UI

Swagger UI can be installed/downloaded on various operating systems, but the best way could be using Docker. A Swagger UI Docker image is available on the Docker Hub. Then we can pass our Open API/Swagger file to the Docker container we run out of the image. Before that, we need to create a JSON file. The Swagger JSON file has few sections:

  • info
  • servers
  • paths

Let's create a Swagger file with the preceding sections for the first service we built. Let's name it openapi.json:

{
"openapi": "3.0.0",
"info": {
"title": "Mirror Finder Service",
"description": "API service for finding the fastest mirror from the
list of given mirror sites",
"version": "0.1.1"
},
"servers": [
{
"url": "http://localhost:8000",
"description": "Development server[Staging/Production are different
from this]"
}
],
"paths": {
"/fastest-mirror": {
"get": {
"summary": "Returns a fastest mirror site.",
"description": "This call returns data of fastest reachable mirror
site",
"responses": {
"200": {
"description": "A JSON object of details",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"fastest_mirror": {
"type": "string"
},
"latency": {
"type": "integer"
}
}
}
}
}
}
}
}
}
}
}

Please notice how we defined the info, servers, and paths sections.

The openapi tag specifies the version of the API document we are using.

The info section has a service-related description. The servers section has the URL of the server where the application/server is running. We used localhost:8000 as we are running it locally. The paths section has information about all the API endpoints a service provides. It also has information about the request body, response type, and body structure. Even the possible error codes can be encapsulated into paths

Now let's install Swagger UI and make use of our Swagger file:

  1. To install Swagger UI via Docker, run this command from your shell:
docker pull swaggerapi/swagger-ui

If you are on Windows 10/Mac OS X , make sure Docker Desktop is running. On Linux, Docker is available all the time once installed.

  1. This pulls the image from the Docker Hub to your local machine. Now we can run a container that can take an openapi.json file and launch Swagger UI. Assuming that you have this file in the chapter1 directory, let's use the following command:
docker run --rm -p 80:8080 -e SWAGGER_JSON=/app/openapi.json -v $GOPATH/github.com/git-user/chapter1:/app swaggerapi/swagger-ui

This command tells Docker to do the following things:

  • Run a container using the swaggerapi/swagger-ui image
  • Mount chapter1 (where openapi.json resides) to the /app directory in the container
  • Expose host port 80 to container port 8080
  • Set the SWAGGER_JSON environment variable to /app/openapi.json

When the container starts, launch http://locahost in the browser. You will see nice documentation for your API:

In this way, without any cost, we can create instant documentation of our REST API using Swagger UI and Open API 3.0.

For testing an API within Swagger UI, the REST API server needs to be accessible to the Docker container (for example, via a network bridge).

From now on, in all chapters, we will try to create Swagger files to document our API design. It is a wise decision to start API development by creating API specifications first and then jumping into implementation. I hope this chapter helped you to brush up on the basics of REST API fundamentals. In the following chapters, we will go deeply into many diverse topics.

 

Summary

In this chapter, we gave an introduction to the REST API. We saw that REST is not a protocol, but an architectural pattern. HTTP is the actual protocol on which we can implement our REST service. We jumped into the fundamentals of the REST API to be clear about what they actually are. Then we explored types of web services. Before REST, we had something called SOAP, which used XML as its data format. REST operates on JSON as its primary format. REST has verbs and status codes. We saw what these status codes refer to.

We designed and implemented a simple service that finds the fastest mirror site to download OS images from all Debian mirror sites hosted worldwide. In this process, we also saw how to package a Go project into a binary. We understood the GOPATH environment variable, which is a workspace definition in Go. We now know that all packages and projects reside on that path.

Next, we jumped into the world of OpenAPI specification by introducing Swagger UI and Swagger files. The structure of these files and how to run Swagger UI with the help of Docker were discussed briefly. We also saw why a developer should start API development by writing down the specifications in the form of a Swagger file.

In the next chapter, we will dig deeper into URL routing. Starting from the built-in router, we will explore Gorilla Mux, a powerful URL routing library.

About the Author

  • Naren Yellavula

    Naren Yellavula, known in the developer community as Naren Arya, started his programming career in a somewhat surprising manner. He ditched mechanical engineering in favor of computer science after watching The Matrix for the first time. With domain expertise in cloud telephony and e-commerce, Naren has a total of 6 years' professional experience and 10 years of programming experience. His articles on open source have been read over a million times worldwide. Naren has spoken at the PyCon India conference on two occasions. He currently works as a software engineer, building microservices for Tradebyte Software GmbH (a Zalando enterprise). In his spare time, he travels to new places. He also loves reading – nonfiction most of the time, and Victorian and Russian fiction on occasion.

    Browse publications by this author

Recommended For You

The Go Workshop

Cut through the noise and get real results with a step-by-step approach to learning Go programming

By Delio D'Anna and 5 more
Hands-On Docker for Microservices with Python

A step-by-step guide to building microservices using Python and Docker, along with managing and orchestrating them with Kubernetes

By Jaime Buelta
Mastering PostgreSQL 12 - Third Edition

Master PostgreSQL 12 features such as advanced indexing, high availability, monitoring, and much more to efficiently manage and maintain your database

By Hans-Jürgen Schönig
The Data Visualization Workshop - Second Edition

Cut through the noise and get real results with a step-by-step approach to learning data visualization with Python

By Mario Döbler and 1 more