Home Web-development Mastering Go Web Services

Mastering Go Web Services

By Nathan Kozyra
books-svg-icon Book
Subscription
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
Subscription
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Our First API in Go
About this book

This book will take you through the most important aspects of designing, building, and deploying a web service utilizing idiomatic REST practices with a focus on speed, security, and flexibility. You will begin by building your first API in Go using the HTTP package. You will look at designing and building your application including popular design structures like Model-View-Controller. You will also understand methods for deploying code to staging and development. Finally, you will see how the security features in Go can be used for protection against SQL injection, and sensitive data compromise.

By the end of this book, you will have achieved a high level of proficiency in building and deploying web services and web APIs with Go.

Publication date:
April 2015
Publisher
Packt
Pages
264
ISBN
9781783981304

 

Chapter 1. Our First API in Go

If you spend any time developing applications on the Web (or off it, for that matter), it won't be long before you find yourself facing the prospect of interacting with a web service or an API.

Whether it's a library that you need or another application's sandbox with which you have to interact, the world of development relies in no small part on the cooperation among dissonant applications, languages, and formats.

That, after all, is why we have APIs to begin with—to allow standardized communication between any two given platforms.

If you spend a long amount of time working on the Web, you'll encounter bad APIs. By bad we mean APIs that are not all-inclusive, do not adhere to best practices and standards, are confusing semantically, or lack consistency. You'll encounter APIs that haphazardly use OAuth or simple HTTP authentication in some places and the opposite in others, or more commonly, APIs that ignore the stated purposes of HTTP verbs (we will discuss more on this later in the chapter).

Google's Go language is particularly well suited to servers. With its built-in HTTP serving, a simple method for XML and JSON encoding of data, high availability, and concurrency, it is the ideal platform for your API.

Throughout this book, we'll not only explore a robust and clean API development but also its interaction with other APIs and data sources, and best practices for such development. We'll build one large service and a bunch of smaller ones for individual, self-contained lessons.

Most importantly, by the end, you should be able to interact with any networked API in Go and be able to design and execute a well-polished API suite yourself.

This book requires at least a casual familiarity with the web-based APIs and a beginner's level competency in Go, but we'll do some very brief introductions when we discuss new concepts and steer you to more information if it turns out that you're not entirely versed in this aspect of either Go or APIs.

We will also touch a bit on concurrency in Go, but we won't get too detailed—if you wish to learn more about this, please check out for the book authored by me, Mastering Concurrency in Go, Packt Publishing.

We will cover the following topics in this chapter:

  • Understanding requirements and dependencies

  • Introducing the HTTP package

  • Building our first routes

  • Setting data via HTTP

  • Serving data from the datastore to the client

 

Understanding requirements and dependencies


Before we get too deep into the weeds in this book, it would be a good idea for us to examine the things that you will need to have installed in order to handle all our examples as we develop, test, and deploy our APIs.

Installing Go

It should go without saying that we will need to have the Go language installed. However, there are a few associated items that you will also need to install in order to do everything we do in this book.

Note

Go is available for Mac OS X, Windows, and most common Linux variants. You can download the binaries at http://golang.org/doc/install.

On Linux, you can generally grab Go through your distribution's package manager. For example, you can grab it on Ubuntu with a simple apt-get install golang command. Something similar exists for most distributions.

In addition to the core language, we'll also work a bit with the Google App Engine, and the best way to test with the App Engine is to install the Software Development Kit (SDK). This will allow us to test our applications locally prior to deploying them and simulate a lot of the functionality that is provided only on the App Engine.

Note

The App Engine SDK can be downloaded from https://developers.google.com/appengine/downloads.

While we're obviously most interested in the Go SDK, you should also grab the Python SDK as there are some minor dependencies that may not be available solely in the Go SDK.

Installing and using MySQL

We'll be using quite a few different databases and datastores to manage our test and real data, and MySQL will be one of the primary ones.

We will use MySQL as a storage system for our users; their messages and their relationships will be stored in our larger application (we will discuss more about this in a bit).

Note

MySQL can be downloaded from http://dev.mysql.com/downloads/.

You can also grab it easily from a package manager on Linux/OS X as follows:

  • Ubuntu: sudo apt-get install mysql-server mysql-client

  • OS X with Homebrew: brew install mysql

Redis

Redis is the first of the two NoSQL datastores that we'll be using for a couple of different demonstrations, including caching data from our databases as well as the API output.

If you're unfamiliar with NoSQL, we'll do some pretty simple introductions to results gathering using both Redis and Couchbase in our examples. If you know MySQL, Redis will at least feel similar, and you won't need the full knowledge base to be able to use the application in the fashion in which we'll use it for our purposes.

Note

Redis can be downloaded from http://redis.io/download.

Redis can be downloaded on Linux/OS X using the following:

  • Ubuntu: sudo apt-get install redis-server

  • OS X with Homebrew: brew install redis

Couchbase

As mentioned earlier, Couchbase will be our second NoSQL solution that we'll use in various products, primarily to set short-lived or ephemeral key store lookups to avoid bottlenecks and as an experiment with in-memory caching.

Unlike Redis, Couchbase uses simple REST commands to set and receive data, and everything exists in the JSON format.

Note

Couchbase can be downloaded from http://www.couchbase.com/download.

  • For Ubuntu (deb), use the following command to download Couchbase:

    dpkg -i couchbase-server version.deb
    

  • For OS X with Homebrew use the following command to download Couchbase:

    brew install https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb
    

Nginx

Although Go comes with everything you need to run a highly concurrent, performant web server, we're going to experiment with wrapping a reverse proxy around our results. We'll do this primarily as a response to the real-world issues regarding availability and speed. Nginx is not available natively for Windows.

Note

  • For Ubuntu, use the following command to download Nginx:

    apt-get install nginx
    

  • For OS X with Homebrew, use the following command to download Nginx:

    brew install nginx
    

Apache JMeter

We'll utilize JMeter for benchmarking and tuning our API for performance. You have a bit of a choice here, as there are several stress-testing applications for simulating traffic. The two we'll touch on are JMeter and Apache's built-in Apache Benchmark (AB) platform. The latter is a stalwart in benchmarking but is a bit limited in what you can throw at your API, so JMeter is preferred.

One of the things that we'll need to consider when building an API is its ability to stand up to heavy traffic (and introduce some mitigating actions when it cannot), so we'll need to know what our limits are.

Note

Apache JMeter can be downloaded from http://jmeter.apache.org/download_jmeter.cgi.

Using predefined datasets

While it's not entirely necessary to have our dummy dataset throughout the course of this book, you can save a lot of time as we build our social network by bringing it in because it is full of users, posts, and images.

By using this dataset, you can skip creating this data to test certain aspects of the API and API creation.

Note

Our dummy dataset can be downloaded at https://github.com/nkozyra/masteringwebservices.

Choosing an IDE

A choice of Integrated Development Environment (IDE) is one of the most personal choices a developer can make, and it's rare to find a developer who is not steadfastly passionate about their favorite.

Nothing in this book will require one IDE over another; indeed, most of Go's strength in terms of compiling, formatting, and testing lies at the command-line level. That said, we'd like to at least explore some of the more popular choices for editors and IDEs that exist for Go.

Eclipse

As one of the most popular and expansive IDEs available for any language, Eclipse is an obvious first mention. Most languages get their support in the form of an Eclipse plugin and Go is no exception.

There are some downsides to this monolithic piece of software; it is occasionally buggy on some languages, notoriously slow for some autocompletion functions, and is a bit heavier than most of the other available options.

However, the pluses are myriad. Eclipse is very mature and has a gigantic community from which you can seek support when issues arise. Also, it's free to use.

Note

Sublime Text

Sublime Text is our particular favorite, but it comes with a large caveat—it is the only one listed here that is not free.

This one feels more like a complete code/text editor than a heavy IDE, but it includes code completion options and the ability to integrate the Go compilers (or other languages' compilers) directly into the interface.

Although Sublime Text's license costs $70, many developers find its elegance and speed to be well worth it. You can try out the software indefinitely to see if it's right for you; it operates as nagware unless and until you purchase a license.

Note

Sublime Text can be downloaded from http://www.sublimetext.com/2.

LiteIDE

LiteIDE is a much younger IDE than the others mentioned here, but it is noteworthy because it has a focus on the Go language.

It's cross-platform and does a lot of Go's command-line magic in the background, making it truly integrated. LiteIDE also handles code autocompletion, go fmt, build, run, and test directly in the IDE and a robust package browser.

It's free and totally worth a shot if you want something lean and targeted directly for the Go language.

Note

LiteIDE can be downloaded from https://code.google.com/p/golangide/.

IntelliJ IDEA

Right up there with Eclipse is the JetBrains family of IDE, which has spanned approximately the same number of languages as Eclipse. Ultimately, both are primarily built with Java in mind, which means that sometimes other language support can feel secondary.

The Go integration here, however, seems fairly robust and complete, so it's worth a shot if you have a license. If you do not have a license, you can try the Community Edition, which is free.

Note

Some client-side tools

Although the vast majority of what we'll be covering will focus on Go and API services, we will be doing some visualization of client-side interactions with our API.

In doing so, we'll primarily focus on straight HTML and JavaScript, but for our more interactive points, we'll also rope in jQuery and AngularJS.

Note

Most of what we do for client-side demonstrations will be available at this book's GitHub repository at https://github.com/nkozyra/goweb under client.

Both jQuery and AngularJS can be loaded dynamically from Google's CDN, which will prevent you from having to download and store them locally. The examples hosted on GitHub call these dynamically.

To load AngularJS dynamically, use the following code:

<script src="//ajax.googleapis.com/ajax/libs/angularjs/1.2.18/angular.min.js"></script>

To load jQuery dynamically, use the following code:

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>

Looking at our application

Throughout this book, we'll be building myriad small applications to demonstrate points, functions, libraries, and other techniques. However, we'll also focus on a larger project that mimics a social network wherein we create and return to users, statuses, and so on, via the API.

Though we'll be working towards the larger application as a way to demonstrate each piece of the puzzle, we'll also build and test self-contained applications, APIs, and interfaces.

The latter group will be prefaced with a quick hitter to let you know that it's not part of our larger application.

Setting up our database

As mentioned earlier, we'll be designing a social network that operates almost entirely at the API level (at least at first) as our master project in this book.

When we think of the major social networks (from the past and in the present), there are a few omnipresent concepts endemic among them, which are as follows:

  • The ability to create a user and maintain a user profile

  • The ability to share messages or statuses and have conversations based on them

  • The ability to express pleasure or displeasure on the said statuses/messages to dictate the worthiness of any given message

There are a few other features that we'll be building here, but let's start with the basics. Let's create our database in MySQL as follows:

create database social_network;

This will be the basis of our social network product in this book. For now, we'll just need a users table to store our individual users and their most basic information. We'll amend this to include more features as we go along:

CREATE TABLE users (
  user_id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  user_nickname VARCHAR(32) NOT NULL,
  user_first VARCHAR(32) NOT NULL,
  user_last VARCHAR(32) NOT NULL,
  user_email VARCHAR(128) NOT NULL,
  PRIMARY KEY (user_id),
  UNIQUE INDEX user_nickname (user_nickname)
)

We won't need to do too much in this chapter, so this should suffice. We'll have a user's most basic information—name, nickname, and e-mail, and not much else.

 

Introducing the HTTP package


The vast majority of our API work will be handled through REST, so you should become pretty familiar with Go's http package.

In addition to serving via HTTP, the http package comprises of a number of other very useful utilities that we'll look at in detail. These include cookie jars, setting up clients, reverse proxies, and more.

The primary entity about which we're interested right now, though, is the http.Server struct, which provides the very basis of all of our server's actions and parameters. Within the server, we can set our TCP address, HTTP multiplexing for routing specific requests, timeouts, and header information.

Go also provides some shortcuts for invoking a server without directly initializing the struct. For example, if you have a lot of default properties, you could use the following code:

Server := Server {
  Addr: ":8080",
  Handler: urlHandler,
  ReadTimeout: 1000 * time.MicroSecond,
  WriteTimeout: 1000 * time.MicroSecond,
  MaxHeaderBytes: 0,
  TLSConfig: nil
}

You can simply execute using the following code:

http.ListenAndServe(":8080", nil)

This will invoke a server struct for you and set only the Addr and Handler properties within.

There will be times, of course, when we'll want more granular control over our server, but for the time being, this will do just fine. Let's take this concept and output some JSON data via HTTP for the first time.

Quick hitter – saying Hello, World via API

As mentioned earlier in this chapter, we'll go off course and do some work that we'll preface with quick hitter to denote that it's unrelated to our larger project.

In this case, we just want to rev up our http package and deliver some JSON to the browser. Unsurprisingly, we'll be merely outputting the uninspiring Hello, world message to, well, the world.

Let's set this up with our required package and imports:

package main

import
(
  "net/http"
  "encoding/json"
  "fmt"
)

This is the bare minimum that we need to output a simple string in JSON via HTTP. Marshalling JSON data can be a bit more complex than what we'll look at here, so if the struct for our message doesn't immediately make sense, don't worry.

This is our response struct, which contains all of the data that we wish to send to the client after grabbing it from our API:

type API struct {
  Message string "json:message"
}

There is not a lot here yet, obviously. All we're setting is a single message string in the obviously-named Message variable.

Finally, we need to set up our main function (as follows) to respond to a route and deliver a marshaled JSON response:

func main() {

  http.HandleFunc("/api", func(w http.ResponseWriter, r *http.Request) {

    message := API{"Hello, world!"}

    output, err := json.Marshal(message)

    if err != nil {
      fmt.Println("Something went wrong!")
    }

    fmt.Fprintf(w, string(output))

  })

  http.ListenAndServe(":8080", nil)
}

Upon entering main(), we set a route handling function to respond to requests at /api that initializes an API struct with Hello, world! We then marshal this to a JSON byte array, output, and after sending this message to our iowriter class (in this case, an http.ResponseWriter value), we cast that to a string.

The last step is a kind of quick-and-dirty approach for sending our byte array through a function that expects a string, but there's not much that could go wrong in doing so.

Go handles typecasting pretty simply by applying the type as a function that flanks the target variable. In other words, we can cast an int64 value to an integer by simply surrounding it with the int(OurInt64) function. There are some exceptions to this—types that cannot be directly cast and some other pitfalls, but that's the general idea. Among the possible exceptions, some types cannot be directly cast to others and some require a package like strconv to manage typecasting.

If we head over to our browser and call localhost:8080/api (as shown in the following screenshot), you should get exactly what we expect, assuming everything went correctly:

 

Building our first route


When we talk about routing in Go nomenclature, we're more accurately discussing a multiplexer or mux. In this case, the multiplexer refers to taking URLs or URL patterns and translating them into internal functions.

You can think of this as a simple mapping from a request to a function (or a handler). You might draw up something like this:

/api/user  func apiUser
/api/message  func apiMessage
/api/status  func apiStatus

There are some limitations with the built-in mux/router provided by the net/http package. You cannot, for example, supply a wildcard or a regular expression to a route.

You might expect to be able to do something as discussed in the following code snippet:

  http.HandleFunc("/api/user/\d+", func(w http.ResponseWriter, r *http.Request) {

    // react dynamically to an ID as supplied in the URL

  })

However, this results in a parsing error.

If you've spent any serious time in any mature web API, you'll know that this won't do. We need to be able to react to dynamic and unpredictable requests. By this we mean that anticipating every numerical user is untenable as it relates to mapping to a function. We need to be able to accept and use patterns.

There are a few solutions for this problem. The first is to use a third-party platform that has this kind of robust routing built in. There are a few very good platforms to choose from, so we'll quickly look at these now.

Gorilla

Gorilla is an all-inclusive web framework, and one that we'll use quite a bit in this book. It has precisely the kind of URL routing package that we need (in its gorilla/mux package), and it also supplies some other very useful tools, such as JSON-RPC, secure cookies, and global session data.

Gorilla's mux package lets us use regular expressions, but it also has some shorthand expressions that let us define the kind of request string we expect without having to write out full expressions.

For example, if we have a request like /api/users/309, we can simple route it as follows in Gorilla:

gorillaRoute := mux.NewRouter()
gorillaRoute.HandleFunc("/api/{user}", UserHandler)

However, there is a clear risk in doing so—by leaving this so open-ended, we have the potential to get some data validation issues. If this function accepts anything as a parameter and we expect digits or text only, this will cause problems in our underlying application.

So, Gorilla allows us to clarify this with regular expressions, which are as follows:

r := mux.NewRouter()
r.HandleFunc("/products/{user:\d+}", ProductHandler)

And now, we will only get what we expect—digit-based request parameters. Let's modify our previous example with this concept to demonstrate this:

package main

import (
  "encoding/json"
  "fmt"
  "github.com/gorilla/mux"
  "net/http"
)

type API struct {
  Message string "json:message"
}

func Hello(w http.ResponseWriter, r *http.Request) {

  urlParams := mux.Vars(r)
  name := urlParams["user"]
  HelloMessage := "Hello, " + name

  message := API{HelloMessage}
  output, err := json.Marshal(message)

  if err != nil {
    fmt.Println("Something went wrong!")
  }

  fmt.Fprintf(w, string(output))

}

func main() {

  gorillaRoute := mux.NewRouter()
  gorillaRoute.HandleFunc("/api/{user:[0-9]+}", Hello)
  http.Handle("/", gorillaRoute)
  http.ListenAndServe(":8080", nil)
}

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

With this code, we have some validation at the routing level. A valid request to /api/44 will give us a proper response, as shown in the following screenshot:

An invalid request to something like /api/nkozyra will give us a 404 response.

Note

Routes

Routes, from drone.io, is explicitly and solely a routing package for Go. This makes it much more focused than the Gorilla web toolkit.

For the most part, URL routing will not be a bottleneck in a smaller application, but it's something that should be considered as your application scales. For our purpose, the differences in speed between, say, Gorilla and Routes is negligible.

Defining your mux package in routes is very clean and simple. Here is a variation on our Hello world message that responds to URL parameters:

func Hello(w http.ResponseWriter, r *http.Request) {

  urlParams := r.URL.Query()
  name := urlParams.Get(":name")
  HelloMessage := "Hello, " + name
  message := API{HelloMessage}
  output, err := json.Marshal(message)

  if err != nil {
    fmt.Println("Something went wrong!")
  }

  fmt.Fprintf(w, string(output))

}

func main() {

  mux := routes.New()
  mux.Get("/api/:name", Hello)
  http.Handle("/", mux)
  http.ListenAndServe(":8080", nil)
}

The primary difference here (as with Gorilla) is that we're passing our routes multiplexer to http instead of using the internal one. And as with Gorilla, we can now use variable URL patterns to change our output, as follows:

Note

You can read more about routes and how to install them at: https://github.com/drone/routes.

Run the following command to install routes:

go get github.com/drone/routes
 

Setting data via HTTP


Now that we've examined how we're going to handle routing, let's take a stab at injecting data into our database directly from a REST endpoint.

In this case, we'll be looking exclusively at the POST request methods because in most cases when large amounts of data could be transferred, you want to avoid the length limitations that the GET requests impose.

Tip

Technically, a PUT request is the semantically correct method to use for requests that are made to create data in the create-read-update-delete (CRUD) concept, but years of disuse have largely relegated PUT to a historical footnote. Recently, some support for restoring PUT (and DELETE) to their proper place has taken hold. Go (and Gorilla) will gladly allow you to relegate requests to either and as we go forward, we'll move towards more protocol-valid semantics.

Connecting to MySQL

Go has a largely built-in agnostic database connection facility, and most third-party database connectivity packages yield to it. Go's default SQL package is database/sql, and it allows more general database connectivity with some standardization.

However, rather than rolling our own MySQL connection (for now, at least), we'll yield to a third-party add-on library. There are a couple of these libraries that are available, but we'll go with Go-MySQL-Driver.

Note

You can install Go-MySQL-Driver using the following command (it requires Git):

go get github.com/go-sql-driver/mysql

For the purpose of this example, we'll assume that you have MySQL running with a localhost on the 3306 standard port. If it is not running, then please make the necessary adjustments accordingly in the examples. The examples here will also use a passwordless root account for the sake of clarity.

Our imports will remain largely the same but with two obvious additions: the sql package (database/sql) and the aforementioned MySQL driver that is imported solely for side effects by prepending it with an underscore:

package main

import
(
  "database/sql"
  _ "github.com/go-sql-driver/mysql"
  "encoding/json"
  "fmt"
  "github.com/gorilla/mux"
  "net/http"
)

We'll set a new endpoint using Gorilla. You may recall that when we intend to set or create data, we'll generally push for a PUT or POST verb, but for the purposes of this demonstration, appending URL parameters is the easiest way to push data. Here is how we'd set up this new route:

  routes := mux.NewRouter()
  routes.HandleFunc("/api/user/create", CreateUser).Methods("GET")

Note

Note that we're specifying the verbs that we'll accept for this request. In real usage, this is recommended for the GET requests.

Our CreateUser function will accept several parameters—user, email, first, and last. User represents a short user name and the rest should be self-explanatory. We'll precede our code with the definition of a User struct as follows:

type User struct {
  ID int "json:id"
  Name  string "json:username"
  Email string "json:email"
  First string "json:first"
  Last  string "json:last"
}

And now let's take a look at the CreateUser function itself:

func CreateUser(w http.ResponseWriter, r *http.Request) {

  NewUser := User{}
  NewUser.Name = r.FormValue("user")
  NewUser.Email = r.FormValue("email")
  NewUser.First = r.FormValue("first")
  NewUser.Last = r.FormValue("last")
  output, err := json.Marshal(NewUser)
  fmt.Println(string(output))
  if err != nil {
    fmt.Println("Something went wrong!")
  }

  sql := "INSERT INTO users set user_nickname='" + NewUser.Name + "', user_first='" + NewUser.First + "', user_last='" + NewUser.Last + "', user_email='" + NewUser.Email + "'"
  q, err := database.Exec(sql)
  if err != nil {
    fmt.Println(err)
  }
  fmt.Println(q)
}

When we run this, our routed API endpoint should be available at localhost:8080/api/user/create. Though if you look at the call itself, you'll note that we need to pass URL parameters to create a user. We're not yet doing any sanity checking on our input, nor are we making certain it's clean/escaped, but we'll hit the URL as follows: http://localhost:8080/api/user/create?user=nkozyra&first=Nathan&last=Kozyra&email=nathan@nathankozyra.com.

And then, we'll end up with a user created in our users table, as follows:

 

Serving data from the datastore to the client


Obviously, if we start setting data via API endpoint—albeit crudely—we'll also want to retrieve the data via another API endpoint. We can easily amend our current call using the following code to include a new route that provides the data back via a request:

func GetUser(w http.ResponseWriter, r *http.Request) {

  urlParams   := mux.Vars(r)
  id       := urlParams["id"]
  ReadUser := User{}
  err := database.QueryRow("select * from users where user_id=?",id).Scan(&ReadUser.ID, &ReadUser.Name, &ReadUser.First, &ReadUser.Last, &ReadUser.Email )
  switch {
      case err == sql.ErrNoRows:
              fmt.Fprintf(w,"No such user")
      case err != nil:
              log.Fatal(err)
  fmt.Fprintf(w, "Error")
      default:
        output, _ := json.Marshal(ReadUser)
        fmt.Fprintf(w,string(output))
  }
}

We're doing a couple of new and noteworthy things here. First, we're using a QueryRow() method instead of Exec(). Go's default database interface offers a couple of different querying mechanisms that do slightly different things. These are as follows:

  • Exec(): This method is used for queries (INSERT, UPDATE, and DELETE primarily) that will not return rows.

  • Query(): This method is used for queries that will return one or more rows. This is usually designated for the SELECT queries.

  • QueryRow(): This method is like Query(), but it expects just one result. This is typically a row-based request similar to the one we had in our previous example. We can then run the Scan() method on that row to inject the returned values into our struct's properties.

Since we're scanning the returned data into our struct, we don't get a return value. With the err value, we run a switch to determine how to convey a response to the user or the application that's using our API.

If we have no rows, it is likely that there is an error in the request and we'll let the recipient know that an error exists.

However, if there is a SQL error, then we'll stay quiet for now. It's a bad practice to expose internal errors to the public. However, we should respond that something went wrong without being too specific.

Finally, if the request is valid and we get a record, we will marshal that into a JSON response and cast it to a string before returning it. Our following result looks like what we'd expect for a valid request:

And then, it appropriately returns an error (as shown in the following screenshot) if we request a particular record from our users' table that does not actually exist:

Setting headers to add detail for clients

Something that will come up a bit more as we go on is the idea of using HTTP headers to convey important information about the data that we're sending or accepting via the API.

We can quickly look at the headers that are being sent through our API now by running a curl request against it. When we do, we'll see something like this:

curl --head http://localhost:8080/api/user/read/1111
HTTP/1.1 200 OK
Date: Wed, 18 Jun 2014 14:09:30 GMT
Content-Length: 12
Content-Type: text/plain; charset=utf-8

This is a pretty small set of headers that is sent by Go, by default. As we go forward, we may wish to append more informative headers that tell a recipient service how to handle or cache data.

Let's very briefly try to set some headers and apply them to our request using the http package. We'll start with one of the more basic response headers and set a Pragma. This is a no-cache Pragma on our result set to tell users or services that ingest our API to always request a fresh version from our database.

Ultimately, given the data we're working with, this is unnecessary in this case, but is the simplest way to demonstrate this behavior. We may find going forward that endpoint caching helps with performance, but it may not provide us with the freshest data.

The http package itself has a pretty simple method for both setting response headers and getting request headers. Let's modify our GetUser function to tell other services that they should not cache this data:

func GetUser(w http.ResponseWriter, r *http.Request) {

  w.Header().Set("Pragma","no-cache")

The Header() method returns the Header struct of iowriter, which we can then add directly using Set() or get by using values using Get().

Now that we've done that, let's see how our output has changed:

curl --head http://localhost:8080/api/user/read/1111
HTTP/1.1 200 OK
Pragma: no-cache
Date: Wed, 18 Jun 2014 14:15:35 GMT
Content-Length: 12
Content-Type: text/plain; charset=utf-8

As we'd expect, we now see our value directly in CURL's header information and it properly returns that this result should not be cached.

There are, of course, far more valuable response headers that we can send with web services and APIs, but this is a good start. As we move forward, we'll utilize more of these, including Content-Encoding, Access-Control-Allow-Origin, and more headers that allow us to specify what our data is, who can access it, and what they should expect in terms of formatting and encoding.

 

Summary


We've touched on the very basics of developing a simple web service interface in Go. Admittedly, this particular version is extremely limited and vulnerable to attack, but it shows the basic mechanisms that we can employ to produce usable, formalized output that can be ingested by other services.

At this point, you should have the basic tools at your disposal that are necessary to start refining this process and our application as a whole. We'll move forward with applying a fuller design to our API as we push forward, as two randomly chosen API endpoints will obviously not do much for us.

In our next chapter, we'll dive in deeper with API planning and design, the nitty-gritty of RESTful services, and look at how we can separate our logic from our output. We'll briefly touch on some logic/view separation concepts and move toward more robust endpoints and methods in Chapter 3, Routing and Bootstrapping.

About the Author
  • Nathan Kozyra

    Nathan Kozyra is a seasoned web developer, with nearly two decades of professional software development experience. Since Go's initial release, he has been drawn to the language for its power, elegance, and usability. He has a strong interest in web development, music production, and machine learning. He is married and has a two-year-old son.

    Browse publications by this author
Latest Reviews (6 reviews total)
Slightly read it and i thought it is great books for people who already have a beginner level of Go at least. But if you don't know much about Go, you better buy with another suitable book(s). The book it self explain detail and straight forward in building web apps.
Easy to read and clear to do the exercises
Good for beginner of the golang. Describe detail of the steps for the developing web services.
Mastering Go Web Services
Unlock this book and the full library FREE for 7 days
Start now