Building RESTful Web services with Go

3.8 (5 reviews total)
By Naren Yellavula
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Getting Started with REST API Development

About this book

REST is an architectural style that tackles the challenges of building scalable web services and in today's connected world, APIs have taken a central role on the web. APIs provide the fabric through which systems interact, and REST has become synonymous with APIs. The depth, breadth, and ease of use of Go, makes it a breeze for developers to work with it to build robust Web APIs. This book takes you through the design of RESTful web services and leverages a framework like Gin
to implement these services.

The book starts with a brief introduction to REST API development and how it transformed the modern web. You will learn how to handle routing and authentication of web services along with working with middleware for internal service. The book explains how to use Go frameworks to build RESTful web services and work with MongoDB to create REST API. You will learn how to integrate Postgres SQL and JSON with a Go web service and build a client library in Go for consuming REST API. You will learn how to scale APIs using the microservice architecture and deploy the REST APIs using Nginx as a proxy server. Finally you will learn how to metricize a REST API using an API Gateway.

By the end of the book you will be proficient in building RESTful APIs in Go.

Publication date:
December 2017
Publisher
Packt
Pages
316
ISBN
9781788294287

 

Chapter 1. Getting Started with REST API Development

A web service is a communication mechanism defined between different computer systems. Without web services, custom peer-to-peer communication becomes cumbersome and platform specific. It is like a hundred different kinds of things that the web needs to understand and interpret. If computer systems align with the protocols that the web can understand easily, it is a great help. A web service is a software system designed to support interoperable machine-to-machine interaction over a network, World Wide Web Consortium (W3C), https://www.w3.org/TR/ws-arch/.

Now, in simple words, a web service is a road between two endpoints where messages are transferred smoothly. Here, this transfer is usually one way. Two individual programmable entities can also communicate with each other through their own APIs. Two people communicate through language. Two applications communicate through the Application Programming Interface (API).

The reader might be wondering; what is the importance of the API in the current digital world? The rise of the Internet of Things (IoT) made API usage heavier than before. Consciousness about the API is growing day by day, and there are hundreds of APIs that are being developed and documented all over the world every day. Notable major businesses are seeing futures in the API as a Service (AAAS). A bright example is Amazon Web Services (AWS). It is a huge success in the cloud world. Developers write their own applications using the REST API provided by the AWS. 

A few more hidden use cases are from travel sites like Ibibo and Expedia, which fetch real-time prices by calling the APIs of third-party gateways and data vendors. Web services are often charged these days.

 

Topics to be covered in this chapter are: 

  • The different Web Services available
  • Representational State Transfer (REST) architecture in detail
  • Introduction to Single Page Applications (SPA) with REST
  • Setting up a Go project and running a development server
  • Building our first service for finding Roman numerals
  • Using Gulp to auto-compile Go code
 

Types of web services


There are many types of web services which have evolved over time. Prominent ones are :

  • SOAP
  • UDDI
  • WSDL
  • REST

Out of these, SOAP became popular in the early 2000s, when XML was on the top wave. The XML data format is used by various distributed systems to communicate with each other. SOAP is too complex to implement. Criticizers of SOAP point out how bulky the SOAP HTTP request is.

A SOAP request usually consists of these three basic components:

  • Envelope
  • Header
  • Body

Just to perform an HTTP request and response cycle, we have to attach a lot of additional data in SOAP. A sample SOAP request looks like this:

POST /StockQuote HTTP/1.1
Host: www.stockquoteserver.com
Content-Type: text/xml; charset="utf-8"
Content-Length: nnnn
SOAPAction: "Some-URI"

<SOAP-ENV:Envelope
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
   <SOAP-ENV:Body>
       <m:GetLastTradePrice xmlns:m="Some-URI">
           <symbol>DIS</symbol>
       </m:GetLastTradePrice>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

This is a standard example of SOAP from the W3C standard (https://www.w3.org/TR/2000/NOTE-SOAP-20000508/). If we observe carefully, it is in XML format, with special tags specifying the envelope and body. Since XML operates on a lot of namespaces to function, additional information comes into play. 

REST API

The name Representational state transfer (REST) was coined by Roy Fielding from the University of California. It is a very simplified and lightweight web service compared to SOAP. Performance, scalability, simplicity, portability, and modifiability are the main principles behind the REST design. 

The REST API allows different systems to communicate and send/receive data in a very simple way. Each and every REST API call has a relation between an HTTP verb and the URL. The resources in the database in an application can be mapped with an API endpoint in the REST.

When you are using a mobile app on your phone, your phone might be secretly talking to many cloud services to retrieve, update, or delete your data. REST services have a huge impact on our daily lives.

REST is a stateless, cacheable, and simple architecture that is not a protocol but a pattern.

Characteristics of REST services

These are the main properties that make REST simple and unique compared to its predecessors:

  • Client-server based architecture: This architecture is most essential for the modern web to communicate over HTTP. A single client-server may look naive initially, but many hybrid architectures are evolving. We will discuss more of these shortly.
  • Stateless: This is the most important characteristic of a REST service. A REST HTTP request consists of all the data needed by the server to understand and give back the response. Once a request is served, the server doesn't remember if the request has arrived after a while. So the operation will be a stateless one.
  • Cacheable: Many developers think a technology stack is blocking their web application or API. But in reality, their architecture is the reason. The database can be a potential tuning piece in a web application. In order to scale an application well, we need to cache content and deliver it as a response. If the cache is not valid, it is our responsibility to bust it. REST services should be properly cached for scaling.
  • Scripts on demand: Have you ever designed a REST service which serves the JavaScript files and you execute them on the fly? This code on demand is also the main characteristic REST can provide. It is more common to request scripts and data from the server.
  • Multiple layered system: The REST API can be served from multiple servers. One server can request the other, and so forth. So when a request comes from the client, request and response can be passed between many servers to finally supply a response back to the client. This easily implementable multi-layered system is always a good strategy for keeping the web application loosely coupled.
  • Representation of resources: The REST API provides the uniform interface to talk to. It uses a Uniform Resource Identifier (URI) to map the resources (data). It also has the advantage of requesting a specific data format as the response. The Internet Media Type (MIME type) can tell the server that the requested resource is of that particular type.
  • Implementational freedom:REST is just a mechanism to define your web services. It is an architectural style that can be implemented in multiple ways. Because of this flexibility, you can create REST services in the way you wish to. Until it follows the principles of REST, your server has the freedom to choose the platform or technology. 

Note

 Thoughtful caching is essential for the REST services to scale.

 

REST verbs and status codes


REST verbs specify an action to be performed on a specific resource or a collection of resources. When a request is made by the client, it should send this information in the HTTP request:

  • REST verb
  • Header information
  • Body (optional)

As we mentioned previously, REST uses the URI to decode its resource to be handled. There are quite a few REST verbs available, but six of them are used frequently. They are as follows:

  • GET
  • POST
  • PUT
  • PATCH
  • DELETE
  • OPTIONS

If you are a software developer, you will be dealing with these six most of the time. The following table explains the operation, target resource, and what happens if the request succeeds or fails:

REST Verb

Action

Success

Failure

GET

Fetches a record or set of resources from the server

200

404

OPTIONS

Fetches all available REST operations 

200

-

POST

Creates a new set of resources or a resource

201

404, 409

PUT

Updates or replaces the given record

200, 204

404

PATCH

Modifies the given record

200, 204

404

DELETE

Deletes the given resource 

200

404

 

The numbers in the Success and Failure columns of the preceding table are HTTP status codes. Whenever a client initiates a REST operation, since REST is stateless, the client should know a way to find out whether the operation was successful or not. For that reason, HTTP has status codes for the response. REST defines the preceding status code types for a given operation. This means a REST API should strictly follow the preceding rules to achieve client-server communication.

All defined REST services have the following format. It consists of the host and API endpoint. The API endpoint is the URL path which is predefined by the server. Every REST request should hit that path.

Note

A trivial REST API URI: http://HostName/API endpoint/Query(optional)

Let us look at all the verbs in more detail. The REST API design starts with the defining of operations and API endpoints. Before implementing the API, the design document should list all the endpoints for the given resources. In the following section, we carefully observe the REST API endpoints using PayPal's REST API as a use case.

GET

A GET method fetches the given resource from the server. To specify a resource, GET uses a few types of URI queries:

  • Query parameters
  • Path-based parameters

In case you didn't know, all of your browsing of the web is done by performing a GET request to the server. For example, if you type www.google.com, you are actually making a GET request to fetch the search page. Here, your browser is the client and Google's web server is the backend implementer of web services. A successful GET operation returns a 200 status code.

Examples of path parameters:

Everyone knows PayPal. PayPal creates billing agreements with companies. If you register with PayPal for a payment system, they provide you with a REST API for all your billing needs. The sample GET request for getting the information of a billing agreement looks like this: /v1/payments/billing-agreements/agreement_id.

Here, the resource query is with the path parameter. When the server sees this line, it interprets it as I got an HTTP request with a need for agreement_id from the billing agreements. Then it searches through the database, goes to the billing-agreements table, and finds an agreement with the given agreement_id. If that resource exists it sends the details to copy back in response (200 OK). Or else it sends a response saying resource not found (404).

Using GET, you can also query a list of resources, instead of a single one like the preceding example. PayPal's API for getting billing transactions related to an agreement can be fetched with /v1/payments/billing-agreements/transactions. This line fetches all transactions that occurred on that billing agreement. In both, the case's data is retrieved in the form of a JSON response. The response format should be designed beforehand so that the client can consume it in the agreement.

Examples of query parameters are as follows:

  • Query parameters are intended to add detailed information to identify a resource from the server. For example, take this sample fictitious API. Let us assume this API is created for fetching, creating, and updating the details of the book. A query parameter based GET request will be in this format: 
      /v1/books/?category=fiction&publish_date=2017
  • The preceding URI has few query parameters. The URI is requesting a book from the book's resource that satisfies the following conditions:
    • It should be a fiction book
    • The book should have been published in the year 2017

Get all the fiction books that are released in the year 2017 is the question the client is posing to the server. 

Path vs Query parameters—When to use them? It is a common rule of thumb that Query parameters are used to fetch multiple resources based on the query parameters. If a client needs a single resource with exact URI information, it can use Path parameters to specify the resource. For example, a user dashboard can be requested with Path parameters and fetch data on filtering can be modeled with Query parameters.

Note

Use Path parameters for a single resource and Query parameters for multiple resources in a GET request.

POST, PUT, and PATCH 

The POST method is used to create a resource on the server. In the previous book's API, this operation creates a new book with the given details. A successful POST operation returns a 201 status code. The POST request can update multiple resources: /v1/books.

The POST request has a body like this: 

{"name" : "Lord of the rings", "year": 1954, "author" : "J. R. R. Tolkien"}

This actually creates a new book in the database. An ID is assigned to this record so that when we GET the resource, the URL is created. So POST should be done only once, in the beginning. In fact, Lord of the Rings was published in 1955. So we entered the published date incorrectly. In order to update the resource, let us use the PUT request.

The PUT method is similar to POST. It is used to replace the resource that already exists. The main difference is that PUT is idempotent. A POST call creates two instances with the same data. But PUT updates a single resource that already exists:

/v1/books/1256

with body that is JSON like this:

{"name" : "Lord of the rings", "year": 1955, "author" : "J. R. R. Tolkien"}

1256 is the ID of the book. It updates the preceding book by year:1955. Did you observe the drawback of PUT? It actually replaced the entire old record with the new one. We needed to change a single column. But PUT replaced the whole record. That is bad. For this reason, the PATCH request was introduced. 

The PATCH method is similar to PUT, except it won't replace the whole record. PATCH, as the name suggests, patches the column that is being modified. Let us update the book 1256 with a new column called ISBN:

/v1/books/1256

with the JSON body like this:

{"isbn" : "0618640150"}

It tells the server, Search for the book with id 1256. Then add/modify this column with the given value.

Note

 PUT and PATCH both return the 200 status for success and 404 for not found.

DELETE and OPTIONS

The DELETE API method is used to delete a resource from the database. It is similar to PUT but without any body. It just needs an ID of the resource to be deleted. Once a resource gets deleted, subsequent GET requests return a 404 not found status. 

Note

Responses to this method are not cacheable (in case caching is implemented)  because the DELETE method is idempotent.

The OPTIONS API method is the most underrated in the API development. Given the resource, this method tries to know all possible methods (GET, POST, and so on) defined on the server. It is like looking at the menu card at a restaurant and then ordering an item which is available (whereas if you randomly order a dish, the waiter will tell you it is not available). It is best practice to implement the OPTIONS method on the server. From the client, make sure OPTIONS is called first, and if the method is available, then proceed with it. 

Cross-Origin Resource Sharing (CORS)

The most important application of this OPTIONS method is Cross-Origin Resource Sharing (CORS). Initially, browser security prevented the client from making cross-origin requests. It means a site loaded with the URL www.foo.com can only make API calls to that host. If the client code needs to request files or data from www.bar.com, then the second server, bar.com, should have a mechanism to recognize foo.com to get its resources.

This process explains the CORS:

  1. foo.com requests the OPTIONS method on bar.com.
  2. bar.com sends a header like Access-Control-Allow-Origin: http://foo.com in response to the client.
  3. Next, foo.com can access the resources on bar.com without any restrictions that call any REST method.

If bar.com feels like supplying resources to any host after one initial request, it can set Access control to * (that is, any).

The following is the diagram depicting the process happening one after the other:

 

Types of status codes

There are a few families of status codes. Each family globally explains an operation status. Each member of that family may have a deeper meeting. So a REST API should strictly tell the client what exactly happened after the operation. There are 60+ status codes available. But for REST, we concentrate on a few families of codes.

2xx family (successful)

200 and 201 fall under the success family. They indicate that an operation was successful. Plain 200 (Operation Successful) is a successful CRUD Operation:

  • 200 (Successful Operation) is the most common type of response status code in REST
  • 201 (Successfully Created) is returned when a POST operation successfully creates a resource on the server
  • 204 (No content) is issued when a client needs a status but not any data back

3xx family (redirection)

These status codes are used to convey redirection messages. The most important ones are 301 and 304:  

  • 301 is issued when a resource is moved permanently to a new URL endpoint. It is essential when an old API is deprecated. It returns the new endpoint in the response with the 301 status. By seeing that, the client should use the new URL in response to achieving its target.
  • The 304 status code indicates that content is cached and no modification happened for the resource on the server. This helps in caching content at the client and only requests data when the cache is modified. 

4xx family (client error)

These are the standard error status codes which the client needs to interpret and handle further actions. These have nothing to do with the server. A wrong request format or ill-formed REST method can cause these errors. Of these, the most frequent status codes API developers use are 400, 401, 403, 404, and 405:

  • 400 (Bad Request) is returned when the server cannot understand the client request.
  • 401 (Unauthorized) is returned when the client is not sending the authorization information in the header.
  • 403 (Forbidden) is returned when the client has no access to a certain type of resources.
  • 404 (Not Found) is returned when the client request is on a resource that is nonexisting.
  • 405 (Method Not Allowed) is returned if the server bans a few methods on resources. GET and HEAD are exceptions.

5xx family (server error)

These are the errors from the server. The client request may be perfect, but due to a bug in the server code, these errors can arise. The commonly used status codes are 500, 501, 502, 503,  and 504:

  • 500 (Internal Server Error) status code gives the development error which is caused by some buggy code or some unexpected condition
  • 501 (Not Implemented) is returned when the server is no longer supporting the method on a resource
  • 502 (Bad Gateway) is returned when the server itself got an error response from another service vendor
  • 503 (Service Unavailable) is returned when the server is down due to multiple reasons, like a heavy load or for maintenance
  • 504 (Gateway Timeout) is returned when the server is waiting a long time for a response from another vendor and is taking too much time to serve the client

For more details on status codes, visit this link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status

 

Rise of REST API with Single Page Applications


You need to understand why Single Page Applications (SPA) are the hot topic today. Instead of building the UI in a traditional way (request web pages), these SPA designs make developers write code in a totally different way. There are many MVC frameworks, like AngularJS, Angular2, React JS, Knockout JS, Aurelia, and so on, to develop web UIs rapidly, but the essence of each of them is pretty simple. All MVC frameworks help us to implement one design pattern. That design pattern is No requesting of web pages, only REST API.

The modern web frontend development has advanced a lot since 2010. In order to exploit the features of Model-View-Controller (MVC) architecture, we need to consider the frontend as a separate entity which talks to the backend only using the REST API (most preferably, REST JSON).

Old and new ways of data flow in SPA

All websites go through the following steps:

  1. Request a web page from the server.
  2. Authenticate and show the Dashboard UI.
  3. Allow the user to modify and save.
  4. Request as many web pages from the server as needed to show individual pages on the site.

But in the SPA, the flow is quite different:

  1. Request the HTML template/s to the browser in one single go.
  2. Then, query the JSON REST API to fill a model (data object).
  3. Adjust the UI according to the data in the model (JSON).
  4. When users modify the UI, the model (data object) should change automatically. For example, in AngularJS, it is possible with two-way data binding. Finally, make REST API calls to notify the server about changes whenever you want.

In this way, communication happens only in the form of the REST API. The client takes care of logically representing the data. This causes systems to move from Response Oriented Architecture (ROA) to Service Oriented Architecture (SOA). Take a look at the following diagram:

 

Note

SPA reduces the bandwidth and improves the site performance.

Why Go for REST API development?

REST services are trivial in the modern web. SOA (which we discuss in more detail later) created an activity space for REST services to take web development to the next level. Go is a programming language from the house of Google for solving the bigger problems they have. It has been eight-plus years since its first appearance. It matured along the way with a developer community jumping in and creating huge scale systems in it.

Note

Go is the darling of the web. It solves bigger problems in an easy way.

One can choose Python or JavaScript (Node) for their REST API development. The main advantage of Go lies in its speed and compile-time error detection. Go is proved to be faster than dynamic programming languages in terms of computational performance by various benchmarks. These are the three reasons why a company should write their next API in Go:

  • To scale your API for a wider audience
  • To enable your developers to build robust systems
  • To invest in the future viability of your projects

You can look at the neverending online debates for more information about REST services with Go. In later chapters, we try to build the fundamentals of designing and writing the REST services.

 

Setting up the project and running the development server


This is a building series book. It assumes you already know the basics of Go. If not, no worries. You can jump start and learn them quickly from Go's official site at https://golang.org/. Go uses a different way of developing projects. Writing a standalone, simple program doesn't bother you much. But after learning the basics, people try to advance a step further. For that reason, as a Go developer, you should know how Go projects are laid out and the best practices to keep your code clean.

Make sure you have done the following things before proceeding:

  • Install Go compiler on your machine
  • Set GOROOT and GOPATH environment variables

There are many online references from which you can get to know the preceding details. Depending on your machine type (Windows, Linux, or macOS X), set up a working Go compiler. We see more details about GOPATH in the following section.

Demystifying GOPATH 

GOPATH is nothing but the current appointed workspace on your machine. It is an environment variable that tells the Go compiler about where your source code, binaries, and packages are placed.

The programmers coming from a Python background may know the Virtualenv tool to create multiple projects (with different Python interpreter versions) at the same time. But at a given time, one activates the environment and develops his project. Similarly,  you can have any number of Go projects on your machine. While developing, set the GOPATH to one of your projects. The Go compiler now activates that project.

It is a common practice to create a project under the home directory and set the GOPATH environment variable like this:

>mkdir /home/naren/myproject
export GOPATH=/home/naren/myproject

Now we install external packages like this:

go get -u -v github.com/gorilla/mux

 Go copies the project called mux into the currently activated project myproject.

Note

For Go get, use the -u flag to install updated dependencies of the external package and -v to see the verbose details of installation.

A typical Go project has the following structure, as mentioned on the official Go website:

                     

 

Let us understand this structure before digging further:

  • bin: Stores the binary of our project; a shippable binary which can be run directly
  • pkg: Contains the package objects; a compiled program which supplies package methods
  • src: The place for your project source code, tests, and user packages

In Go, all the packages which you import into your main program have an identical structure, github.com/user/project. But who creates all these directories? Should the developer do that? Nope. It is the developer's responsibility to create directories for his/her project. It means he/she only creates the directory src/github.com/user/hello.

When a developer runs the following command, the directories bin and package are created if they did not exist before. .bin consists of the binary of our project source code and .pkg consists of all internal and external packages we use in our Go programs:

 go install github.com/user/project
 

Building our first service – finding the Roman numeral


With the concepts we have built upto now, let us write our first basic REST service. This service takes the number range (1-10) from the client and returns its Roman string. Very primitive, but better than Hello World.

Design:

Our REST API should take an integer number from the client and serve back the Roman equivalent.

The block of the API design document may look like this:

HTTP Verb

PATH

Action

Resource

GET

/roman_number/2

show

roman_number

 

Implementation:

Now we are going to implement the preceding simple API step-by-step.

Note

Code for this project is available at https://github.com/narenaryan/gorestful

As we previously discussed, you should set the GOPATH first. Let us assume the GOPATH is /home/naren/go. Create a directory called romanserver in the following path. Replace narenaryan with your GitHub username (this is just a namespace for the code belonging to different users):

mkdir -p $GOPATH/src/github.com/narenaryan/romanserver

Our project is ready. We don't have any database configured yet. Create an empty file called main.go:

touch $GOPATH/src/github.com/narenaryan/romanserver/main.go

Our main logic for the API server goes into this file. For now, we can create a data file which works as a data service for our main program. Create one more directory for packaging the Roman numeral data: 

mkdir $GOPATH/src/github.com/narenaryan/romanNumerals

Now, create an empty file called data.go in the romanNumerals directory.  The src directory structure so far looks like this:

 ;

Now let us start adding code to the files. Create data for the Roman numerals:

// data.go
package romanNumerals

var Numerals = map[int]string{
  10: "X",
  9: "IX",
  8: "VIII",
  7: "VII",
  6: "VI",
  5: "V",
  4: "IV",
  3: "III",
  2: "II",
  1: "I",
}

We are creating a map called Numerals. This map holds information for converting a given integer to its Roman equivalent. We are going to import this variable into our main program to serve the request from the client. 

Open main.go and add the following code:

// main.go
package main

import (
   "fmt"
   "github.com/narenaryan/romanNumerals"
   "html"
   "net/http"
   "strconv"
   "strings"
   "time"
)

func main() {
   // http package has methods for dealing with requests
   http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
       urlPathElements := strings.Split(r.URL.Path, "/")
       // If request is GET with correct syntax
       if urlPathElements[1] == "roman_number" {
           number, _ := strconv.Atoi(strings.TrimSpace(urlPathElements[2]))
           if number == 0 || number > 10 {
           // If resource is not in the list, send Not Found status
               w.WriteHeader(http.StatusNotFound)
               w.Write([]byte("404 - Not Found"))
           } else {
             fmt.Fprintf(w, "%q", html.EscapeString(romanNumerals.Numerals[number]))
           }
       } else {
           // For all other requests, tell that Client sent a bad request
           w.WriteHeader(http.StatusBadRequest)
           w.Write([]byte("400 - Bad request"))
       }
   })
 // Create a server and run it on 8000 port
   s := &http.Server{
     Addr: ":8000",
     ReadTimeout: 10 * time.Second,
     WriteTimeout: 10 * time.Second,
     MaxHeaderBytes: 1 << 20,
   }
   s.ListenAndServe()
}

Note

Always use the Go fmt tool to format your Go code. Usage example: go fmt github.com/narenaryan/romanserver

Now, install this project with the Go command install:

go install github.com/narenaryan/romanserver

This step does two things:

  • Compiles the package romanNumerals and places a copy in the $GOPATH/pkg directory
  • Places a binary in the $GOPATH/bin

We can run the preceding API server as this:

$GOPATH/bin/romanserver

The server is up and running on http://localhost:8000. Now we can make a GET request to the API using a client like Browser or the CURL command. Let us fire a CURL command with a proper API GET request.

Request one is as follows:

curl -X GET "http://localhost:8000/roman_number/5" # Valid request

The response is as follows:

HTTP/1.1 200 OK
Date: Sun, 07 May 2017 11:24:32 GMT
Content-Length: 3
Content-Type: text/plain; charset=utf-8

"V"

Let us try a few incorrectly formed requests.

Request two is as follows:

curl -X GET "http://localhost:8000/roman_number/12" # Resource out of range

The response is as follows: 

HTTP/1.1 404 Not Found
Date: Sun, 07 May 2017 11:22:38 GMT
Content-Length: 15
Content-Type: text/plain; charset=utf-8

404 - Not Found

Request three is as follows:

curl -X GET "http://localhost:8000/random_resource/3" # Invalid resource

The response is as follows:

"HTTP/1.1 400 Bad request
Date: Sun, 07 May 2017 11:22:38 GMT
Content-Length: 15
Content-Type: text/plain; charset=utf-8
400 - Bad request

Our little Roman numerals API is doing the right thing. The right status codes are being returned. That is the point all API developers should keep in mind. The client should be informed why something went wrong.

Breaking down the code

We just updated the empty files in one single go and started running the server. Let me now explain each and every piece of the file main.go:

  • Imported a few packages. github.com/narenaryan/romanNumerals is the data service we created before.
  • net/http is the core package we used to handle an HTTP request through its HandleFunc function. That function's arguments are http.Request and http.ResponseWriter. Those two deal with the request and response of an HTTP request.
  • r.URL.Path is the URL path of the HTTP request. For the CURL Request one, it is /roman_number/5. We are splitting this path and using the second argument as a resource and the third argument as a value to get the Roman numeral. The Split function is in a core package called strings.
  • The Atoi function converts an alphanumeric string to an integer. For the numerals map to consume, we need to convert the integer string to an integer. The Atoi function comes from a core package called strconv.
  •  We use http.StatusXXX to set the status code of the response header. The WriteHeader and Write functions are available on the response object for writing the header and body, respectively.
  • Next, we created an HTTP server using &http while initializing a few parameters like address, port, timeout, and so on.
  • The time package is used to define seconds in the program. It says, after 10 seconds of inactivity, automatically return a 408 request timeout back to the client.
  • EscapeString escapes special characters to become valid HTML characters. For example, Fran & Freddie's becomes Fran &amp; Freddie's&#34.
  • Finally, start the server with the  ListenAndServe function. It keeps your web server running until you kill it.

Note

One should write unit tests for their API. In the upcoming chapters, we will see how to test an API end to end. 

Live reloading the application with supervisord and Gulp

Gulp is a nice tool for creating workflows. A workflow is a step-by-step process. It is nothing but a task streamlining application. You need NPM and Node installed on your machine. We use Gulp to watch the files and then update the binary and restart the API server. Sounds cool, right?

The supervisor is an application to reload your server whenever the application gets killed. A process ID will be assigned to your server. To restart the app properly, we need to kill the existing instances and restart the application. We can write one such program in Go. But in order to not reinvent the wheel, we are using a popular program called supervisord.  

Monitoring your Go web server with supervisord

Sometimes your web application may stop due to operating system restarts or crashes. Whenever your web server gets killed, it is supervisor's job to bring it back to life. Even the system restart cannot take your web server away from the customers. So, strictly use supervisord for your app monitoring.

Installing supervisord

We can easily install supervisord on Ubuntu 16.04, with the apt-get command:

sudo apt-get install -y supervisor

This installs two tools, supervisor and supervisorctl. supervisorctl is intended to control the supervisord and add tasks, restart tasks, and so on.

On macOS X, we can install supervisor using the brew command:

brew install supervisor

Now, create a configuration file at:

/etc/supervisor/conf.d/goproject.conf

You can add any number of configuration files, and supervisord treats them as separate processes to run. Add the following content to the preceding file:

[supervisord]
logfile = /tmp/supervisord.log
[program:myserver]
command=$GOPATH/bin/romanserver
autostart=true
autorestart=true
redirect_stderr=true

By default, we have a file called .supervisord.conf at /etc/supervisor/. Look at it for more reference. In macOS X, the same file will be located at /usr/local/etc/supervisord.ini.

Coming to the preceding configuration:

  • The [supervisord] section tells the location of the log file for supervisord
  • [program:myserver] is the task block which traverses to the given directory and executes the command given

Now we can ask our supervisorctl to re-read the configuration and restart the tasks (process). For that, just say:

  • supervisorctl reread
  • supervisorctl update

Then, launch supervisorctl with the command:

supervisorctl

You will see something like this:

Note

supervisorctl is a great tool for controlling supervisor programs.

   

Since we named our romanserver myserver in the supervisor configuration file, we can start, stop, and restart that program from supervisorctl.

 

Using Gulp for creating auto code compiling and server reloading


With the little introduction we gave about Gulp in the preceding section, we are going to write a gulpfile for telling the computer to execute a few tasks.

I install Gulp and Gulp-shell using npm:

npm install gulp gulp-shell

After this, create a gulpfile.js in the root directory of the project. Here, it is github.com/src/narenaryan/romanserver. Now add this content to gulpfile.js. First, whenever a file changes, install binary task gets executed. Then, the supervisor will be restarted. The watch task looks for any file change and executes the preceding tasks. We are also ordering the tasks so that they occur one after the other synchronously. All of these tasks are Gulp tasks and can be defined by the gulp.task function. It takes two arguments with task name, task. sell.task allows Gulp to execute system commands:

var gulp = require("gulp");
var shell = require('gulp-shell');

// This compiles new binary with source change
gulp.task("install-binary", shell.task([
 'go install github.com/narenaryan/romanserver'
]));

// Second argument tells install-binary is a deapendency for restart-supervisor
gulp.task("restart-supervisor", ["install-binary"], shell.task([
 'supervisorctl restart myserver'
]))

gulp.task('watch', function() {
 // Watch the source code for all changes
 gulp.watch("*", ['install-binary', 'restart-supervisor']);

});

gulp.task('default', ['watch']);

Now, if you run the  gulp command in the source directory, it starts watching your source code changes:

gulp

Now, if we modify the code, then the code is compiled, installed, and the server restarted in a flash:

 

Understanding the gulpfile

In the gulpfile, we are performing the following instructions:

  1. Import Gulp and Gulp-shell.
  2. Create tasks with shell.task as the function to execute.
  3. shell.task can execute a command-line instruction. Keep your shell commands inside that function.
  4. Add a watch task for watching source files. The task list will be executed when files are modified.
  5. Create a default task for running. Add a watch to it.

Gulp is a great tool for these kinds of use cases. So, please go through the official documentation of Gulp at http://gulpjs.com/.

 

Summary


In this chapter, we gave an introduction to the REST API. We saw that REST is not a protocol, but an architectural pattern. HTTP is the actual protocol on which we can implement our REST service. We jumped into the fundamentals of the REST API to be clear about what they actually are. Then we explored types of web services. Before REST, we have something called SOAP, which uses XML as the data format. REST operates on JSON as the primary format. REST has verbs and status codes. We saw what a given status code refers to. We built a simple service which serves the Roman numerals for given numbers. In this process, we also saw how to package a Go project. We understood the GOPATH environment variable. It is a workspace defining a variable in Go. All packages and projects reside in that path. We then saw how to reload a development project on the fly with the help of supervisord and Gulp. These are node tools but can help us to keep our Go project up and running.

In the next chapter, we dig deeper into URL routing. Starting from the built-in router, we explore Gorilla Mux, a powerful URL routing library.

About the Author

  • Naren Yellavula

    Naren Yellavula—fondly called by the developer community as Naren Arya—started his programming career in the funniest way. He ditched mechanical engineering for computer science after watching The Matrix for the first time. With dreams of coding a mini world, he continuously sharpened his practical skills. Initially, he built few a mobile applications. Then, he completely moved into the space of full stack development. He always believed that computers and software can help make this world a better place. He wrote 120+ development articles on various open source technologies, including Go and Python. He is also one of the few young speakers at the PyCon India conference (he spoke at Bangalore in 2015 and Delhi in 2017). With detailed knowledge of web scraping and REST APIs, he dived into multiple domains such as cloud telephony and cloud-based web services. On this journey, he bathed in a lot of design decisions. He currently works as a software engineer for Citrix R&D, India. Naren is a great fan of Go personally because of the speed and intuitiveness of the language. In his spare time, he tries to educate the youth in programming and software development. He loves reading nonfiction most of the time, and Victorian and Russian fiction sometimes.

    Browse publications by this author

Latest Reviews

(5 reviews total)
Never can make the payment, therefore I can not purchase anything.
good and easy tutorial about go
RESTful api is core part of our mobile architecture with microservcies. Excellent and usefull book

Recommended For You

Book Title
Unlock this book and the full library for FREE
Start free trial