Download code from GitHub
Go is great for writing high-performance, concurrent server applications and tools, and the Web is the perfect medium over which to deliver them. It would be difficult these days to find a gadget that is not web-enabled and this allows us to build a single application that targets almost all platforms and devices.
Our first project will be a web-based chat application that allows multiple users to have a real-time conversation right in their web browser. Idiomatic Go applications are often composed of many packages, which are organized by having code in different folders, and this is also true of the Go standard library. We will start by building a simple web server using the net/http package, which will serve the HTML files. We will then go on to add support for web sockets through which our messages will flow.
In languages such as C#, Java, or Node.js, complex threading code and clever use of locks need to be employed in order to keep all clients in sync. As we will see, Go helps us enormously with its built-in channels and concurrency paradigms.
In this chapter, you will learn how to:
Use the
net/httppackage to serve HTTP requestsDeliver template-driven content to users' browsers
Satisfy a Go interface to build our own
http.HandlertypesUse Go's goroutines to allow an application to perform multiple tasks concurrently
Use channels to share information between running goroutines
Upgrade HTTP requests to use modern features such as web sockets
Add tracing to the application to better understand its inner working
Write a complete Go package using test-driven development practices
Return unexported types through exported interfaces
Note
Complete source code for this project can be found at https://github.com/matryer/goblueprints/tree/master/chapter1/chat. The source code was periodically committed so the history in GitHub actually follows the flow of this chapter too.
The first thing our chat application needs is a web server that has two main responsibilities:
Serving the HTML and JavaScript chat clients that run in the user's browser
Accepting web socket connections to allow the clients to communicate
Note
The GOPATH environment variable is covered in detail in Appendix, Good Practices for a Stable Go environment. Be sure to read that first if you need help getting set up.
Create a main.go file inside a new folder called chat in your GOPATH and add the following code:
package main
import (
"log"
"net/http"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`
<html>
<head>
<title>Chat</title>
</head>
<body>
Let's chat!
</body>
</html>
))
})
// start the web server
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
This is a complete, albeit simple, Go program that will:
Listen to the root path using the
net/httppackageWrite out the hardcoded HTML when a request is made
Start a web server on port
:8080using theListenAndServemethod
The http.HandleFunc function maps the path pattern / to the function we pass as the second argument, so when the user hits http://localhost:8080/, the function will be executed. The function signature of func(w http.ResponseWriter, r *http.Request) is a common way of handling HTTP requests throughout the Go standard library.
Tip
We are using package main because we want to build and run our program from the command line. However, if we were building a reusable chatting package, we might choose to use something different, such as package chat.
In a terminal, run the program by navigating to the main.go file you just created and execute the following command:
go run main.go
Tip
The go run command is a helpful shortcut for running simple Go programs. It builds and executes a binary in one go. In the real world, you usually use go build yourself to create and distribute binaries. We will explore this later.
Open the browser and type http://localhost:8080 to see the Let's chat! message.
Having the HTML code embedded within our Go code like this works, but it is pretty ugly and will only get worse as our projects grow. Next, we will see how templates can help us clean this up.
Templates allow us to blend generic text with specific text, for instance, injecting a user's name into a welcome message. For example, consider the following template:
Hello {{name}}, how are you?
We are able to replace the {{name}} text in the preceding template with the real name of a person. So if Bruce signs in, he might see:
Hello Bruce, how are you?
The Go standard library has two main template packages: one called text/template for text and one called html/template for HTML. The html/template package does the same as the text version except that it understands the context in which data will be injected into the template. This is useful because it avoids script injection attacks and resolves common issues such as having to encode special characters for URLs.
Initially, we just want to move the HTML code from inside our Go code to its own file, but won't blend any text just yet. The template packages make loading external files very easy, so it's a good choice for us.
Create a new folder under our chat folder called templates and create a chat.html file inside it. We will move the HTML from main.go to this file, but we will make a minor change to ensure our changes have taken effect:
<html>
<head>
<title>Chat</title>
</head>
<body>
Let's chat (from template)
</body>
</html>
Now, we have our external HTML file ready to go, but we need a way to compile the template and serve it to the user's browser.
Tip
Compiling a template is a process by which the source template is interpreted and prepared for blending with various data, which must happen before a template can be used but only needs to happen once.
We are going to write our own struct type that is responsible for loading, compiling, and delivering our template. We will define a new type that will take a filename string, compile the template once (using the sync.Once type), keep the reference to the compiled template, and then respond to HTTP requests. You will need to import the text/template, path/filepath, and sync packages in order to build your code.
In main.go, insert the following code above the func main() line:
// templ represents a single template
type templateHandler struct {
once sync.Once
filename string
templ *template.Template
}
// ServeHTTP handles the HTTP request.
func (t *templateHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
t.once.Do(func() {
t.templ = template.Must(template.ParseFiles(filepath.Join("templates",
t.filename)))
})
t.templ.Execute(w, nil)
}
Tip
Did you know that you could automate the adding and removing of imported packages? See Appendix, Good Practices for a Stable Go Environment, on how to do this.
The templateHandler type has a single method called ServeHTTP whose signature looks suspiciously like the method we passed to http.HandleFunc earlier. This method will load the source file, compile the template and execute it, and write the output to the specified http.ResponseWriter method. Because the ServeHTTP method satisfies the http.Handler interface, we can actually pass it directly to http.Handle.
Tip
A quick look at the Go standard library source code, which is located at
http://golang.org/pkg/net/http/#Handler, will reveal that the interface definition for http.Handler specifies that only the ServeHTTP method need be present in order for a type to be used to serve HTTP requests by the net/http package.
We only need to compile the template once, and there are a few different ways to approach this in Go. The most obvious is to have a NewTemplateHandler function that creates the type and calls some initialization code to compile the template. If we were sure the function would be called by only one goroutine (probably the main one during the setup in the main function), this would be a perfectly acceptable approach. An alternative, which we have employed in the preceding section, is to compile the template once inside the ServeHTTP method. The sync.Once type guarantees that the function we pass as an argument will only be executed once, regardless of how many goroutines are calling ServeHTTP. This is helpful because web servers in Go are automatically concurrent and once our chat application takes the world by storm, we could very well expect to have many concurrent calls to the ServeHTTP method.
Compiling the template inside the ServeHTTP method also ensures that our code does not waste time doing work before it is definitely needed. This lazy initialization approach doesn't save us much in our present case, but in cases where the setup tasks are time- and resource-intensive and where the functionality is used less frequently, it's easy to see how this approach would come in handy.
To implement our templateHandler type, we need to update the main body function so that it looks like this:
func main() {
// root
http.Handle("/", &templateHandler{filename: "chat.html"})
// start the web server
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
The templateHandler structure is a valid http.Handler type so we can pass it directly to the http.Handle function and ask it to handle requests that match the specified pattern. In the preceding code, we created a new object of the type templateHandler, specifying the filename as chat.html that we then take the address of (using the & address of the operator) and pass it to the http.Handle function. We do not store a reference to our newly created templateHandler type, but that's OK because we don't need to refer to it again.
In your terminal, exit the program by pressing Ctrl + C and re-run it, then refresh your browser and notice the addition of the (from template) text. Now our code is much simpler than an HTML code and free from its ugly blocks.
Running Go programs using a go run command is great when our code is made up of a single main.go file. However, often we might quickly need to add other files. This requires us to properly build the whole package into an executable binary before running it. This is simple enough, and from now on, this is how you will build and run your programs in a terminal:
go build -o {name} ./{name}
The go build command creates the output binary using all the .go files in the specified folder, and the -o flag indicates the name of the generated binary. You can then just run the program directly by calling it by name.
For example, in the case of our chat application, we could run:
go build -o chat ./chat
Since we are compiling templates the first time the page is served, we will need to restart your web server program every time anything changes in order to see the changes take effect.
All users (clients) of our chat application will automatically be placed in one big public room where everyone can chat with everyone else. The room type will be responsible for managing client connections and routing messages in and out, while the client type represents the connection to a single client.
To manage our web sockets, we are going to use one of the most powerful aspects of the Go community open source third-party packages. Every day, new packages solving real-world problems are released, ready for you to use in your own projects, and they even allow you to add features, report and fix bugs, and get support.
Tip
It is often unwise to reinvent the wheel unless you have a very good reason. So before embarking on building a new package, it is worth searching for any existing projects that might have already solved your very problem. If you find one similar project that doesn't quite satisfy your needs, consider contributing to the project and adding features. Go has a particularly active open source community (remember that Go itself is open source) that is always ready to welcome new faces or avatars.
We are going to use Gorilla Project's websocket package to handle our server-side sockets rather than write our own. If you're curious about how it works, head over to the project home page on GitHub, https://github.com/gorilla/websocket, and browse the open source code.
Create a new file called client.go alongside main.go in the chat folder and add the following code:
package main
import (
"github.com/gorilla/websocket"
)
// client represents a single chatting user.
type client struct {
// socket is the web socket for this client.
socket *websocket.Conn
// send is a channel on which messages are sent.
send chan []byte
// room is the room this client is chatting in.
room *room
}
In the preceding code, socket will hold a reference to the web socket that will allow us to communicate with the client, and the send field is a buffered channel through which received messages are queued ready to be forwarded to the user's browser (via the socket). The room field will keep a reference to the room that the client is chatting in this is required so that we can forward messages to everyone else in the room.
If you try to build this code, you will notice a few errors. You must ensure that you have called go get to retrieve the websocket package, which is as easy as opening a terminal and typing the following:
go get github.com/gorilla/websocket
Building the code again will yield another error:
./client.go:17 undefined: room
The problem is that we have referred to a room type without defining it anywhere. To make the compiler happy, create a file called room.go and insert the following placeholder code:
package main
type room struct {
// forward is a channel that holds incoming messages
// that should be forwarded to the other clients.
forward chan []byte
}
We will improve this definition later once we know a little more about what our room needs to do, but for now, this will allow us to proceed. Later, the forward channel is what we will use to send the incoming messages to all other clients.
Note
You can think of channels as an in-memory thread-safe message queue where senders pass data and receivers read data in a non-blocking, thread-safe way.
In order for a client to do any work, we must define some methods that will do the actual reading and writing to and from the web socket. Adding the following code to client.go outside (underneath) the client struct will add two methods called read and write to the client type:
func (c *client) read() {
defer c.socket.Close()
for {
_, msg, err := c.socket.ReadMessage()
if err != nil {
return
}
c.room.forward <- msg
}
}
func (c *client) write() {
defer c.socket.Close()
for msg := range c.send {
err := c.socket.WriteMessage(websocket.TextMessage, msg)
if err != nil {
return
}
}
}
The read method allows our client to read from the socket via the ReadMessage method, continually sending any received messages to the forward channel on the room type. If it encounters an error (such as 'the socket has died'), the loop will break and the socket will be closed. Similarly, the write method continually accepts messages from the send channel writing everything out of the socket via the WriteMessage method. If writing to the socket fails, the for loop is broken and the socket is closed. Build the package again to ensure everything compiles.
Note
In the preceding code, we introduced the defer keyword, which is worth exploring a little. We are asking Go to run c.socket.Close() when the function exits. It's extremely useful for when you need to do some tidying up in a function (such as closing a file or, as in our case, a socket) but aren't sure where the function will exit. As our code grows, if this function has multiple return statements, we won't need to add any more calls to close the socket, because this single defer statement will catch them all.
Some people complain about the performance of using the defer keyword, since it doesn't perform as well as typing the close statement before every exit point in the function. You must weigh up the runtime performance cost against the code maintenance cost and potential bugs that may get introduced if you decide not to use defer. As a general rule of thumb, writing clean and clear code wins; after all, we can always come back and optimize any bits of code we feel is slowing our product down if we are lucky enough to have such success.
We need a way for clients to join and leave rooms in order to ensure that the c.room.forward <- msg code in the preceding section actually forwards the message to all the clients. To ensure that we are not trying to access the same data at the same time, a sensible approach is to use two channels: one that will add a client to the room and another that will remove it. Let's update our room.go code to look like this:
package main
type room struct {
// forward is a channel that holds incoming messages
// that should be forwarded to the other clients.
forward chan []byte
// join is a channel for clients wishing to join the room.
join chan *client
// leave is a channel for clients wishing to leave the room.
leave chan *client
// clients holds all current clients in this room.
clients map[*client]bool
}
We have added three fields: two channels and a map. The join and leave channels exist simply to allow us to safely add and remove clients from the clients map. If we were to access the map directly, it is possible that two goroutines running concurrently might try to modify the map at the same time, resulting in corrupt memory or unpredictable state.
Now we get to use an extremely powerful feature of Go's concurrency offerings the select statement. We can use select statements whenever we need to synchronize or modify shared memory, or take different actions depending on the various activities within our channels.
Beneath the room structure, add the following run method that contains three select cases:
func (r *room) run() {
for {
select {
case client := <-r.join:
// joining
r.clients[client] = true
case client := <-r.leave:
// leaving
delete(r.clients, client)
close(client.send)
case msg := <-r.forward:
// forward message to all clients
for client := range r.clients {
client.send <- msg
}
}
}
}
Although this might seem like a lot of code to digest, once we break it down a little, we will see that it is fairly simple, although extremely powerful. The top for loop indicates that this method will run forever, until the program is terminated. This might seem like a mistake, but remember, if we run this code as a goroutine, it will run in the background, which won't block the rest of our application. The preceding code will keep watching the three channels inside our room: join, leave, and forward. If a message is received on any of those channels, the select statement will run the code for that particular case.
Note
It is important to remember that it will only run one block of case code at a time. This is how we are able to synchronize to ensure that our r.clients map is only ever modified by one thing at a time.
If we receive a message on the join channel, we simply update the r.clients map to keep a reference of the client that has joined the room. Notice that we are setting the value to true. We are using the map more like a slice, but do not have to worry about shrinking the slice as clients come and go through time setting the value to true is just a handy, low-memory way of storing the reference.
If we receive a message on the leave channel, we simply delete the client type from the map, and close its send channel. If we receive a message on the forward channel, we iterate over all the clients and add the message to each client's send channel. Then, the write method of our client type will pick it up and send it down the socket to the browser.
Now we are going to turn our room type into an http.Handler type like we did with the template handler earlier. As you will recall, to do this, we must simply add a method called ServeHTTP with the appropriate signature.
Add the following code to the bottom of the room.go file:
const (
socketBufferSize = 1024
messageBufferSize = 256
)
var upgrader = &websocket.Upgrader{ReadBufferSize: socketBufferSize,
WriteBufferSize: socketBufferSize}
func (r *room) ServeHTTP(w http.ResponseWriter, req *http.Request) {
socket, err := upgrader.Upgrade(w, req, nil)
if err != nil {
log.Fatal("ServeHTTP:", err)
return
}
client := &client{
socket: socket,
send: make(chan []byte, messageBufferSize),
room: r,
}
r.join <- client
defer func() { r.leave <- client }()
go client.write()
client.read()
}
The ServeHTTP method means a room can now act as a handler. We will implement it shortly, but first let's have a look at what is going on in this snippet of code.
Tip
If you accessed the chat endpoint in a web browser, you would likely crash the program and see an error like ServeHTTPwebsocket: version != 13. This is because it is intended to be accessed via a web socket rather than a web browser.
In order to use web sockets, we must upgrade the HTTP connection using the websocket.Upgrader type, which is reusable so we need only create one. Then, when a request comes in via the ServeHTTP method, we get the socket by calling the upgrader.Upgrade method. All being well, we then create our client and pass it into the join channel for the current room. We also defer the leaving operation for when the client is finished, which will ensure everything is tidied up after a user goes away.
The write method for the client is then called as a goroutine, as indicated by the three characters at the beginning of the line go (the word go followed by a space character). This tells Go to run the method in a different thread or goroutine.
Note
Compare the amount of code needed to achieve multithreading or concurrency in other languages with the three key presses that achieve it in Go, and you will see why it has become a favorite among system developers.
Finally, we call the read method in the main thread, which will block operations (keeping the connection alive) until it's time to close it. Adding constants at the top of the snippet is a good practice for declaring values that would otherwise be hardcoded throughout the project. As these grow in number, you might consider putting them in a file of their own, or at least at the top of their respective files so they remain easy to read and modify.
Our room is almost ready to go, although in order for it to be of any use, the channels and map need to be created. As it is, this could be achieved by asking the developer to use the following code to be sure to do this:
r := &room{
forward: make(chan []byte),
join: make(chan *client),
leave: make(chan *client),
clients: make(map[*client]bool),
}
Another, slightly more elegant, solution is to instead provide a newRoom function that does this for us. This removes the need for others to know about exactly what needs to be done in order for our room to be useful. Underneath the type room struct definition, add this function:
// newRoom makes a new room.
func newRoom() *room {
return &room{
forward: make(chan []byte),
join: make(chan *client),
leave: make(chan *client),
clients: make(map[*client]bool),
}
}
Now the users of our code need only call the newRoom function instead of the more verbose six lines of code.
Let's update our main function in main.go to first create and then run a room for everybody to connect to:
func main() {
r := newRoom()
http.Handle("/", &templateHandler{filename: "chat.html"})
http.Handle("/room", r)
// get the room going
go r.run()
// start the web server
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
We are running the room in a separate goroutine (notice the go keyword again) so that the chatting operations occur in the background, allowing our main goroutine to run the web server. Our server is now finished and successfully built, but remains useless without clients to interact with.
In order for the users of our chat application to interact with the server and therefore other users, we need to write some client-side code that makes use of the web sockets found in modern browsers. We are already delivering HTML content via the template when users hit the root of our application, so we can enhance that.
Update the chat.html file in the templates folder with the following markup:
<html>
<head>
<title>Chat</title>
<style>
input { display: block; }
ul { list-style: none; }
</style>
</head>
<body>
<ul id="messages"></ul>
<form id="chatbox">
<textarea></textarea>
<input type="submit" value="Send" />
</form> </body>
</html>
The preceding HTML will render a simple web form on the page containing a text area and a Send button this is how our users will submit messages to the server. The messages element in the preceding code will contain the text of the chat messages so that all the users can see what is being said. Next, we need to add some JavaScript to add some functionality to our page. Underneath the form tag, above the closing </body> tag, insert the following code:
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"> </script>
<script>
$(function(){
var socket = null;
var msgBox = $("#chatbox textarea");
var messages = $("#messages");
$("#chatbox").submit(function(){
if (!msgBox.val()) return false;
if (!socket) {
alert("Error: There is no socket connection.");
return false;
}
socket.send(msgBox.val());
msgBox.val("");
return false;
});
if (!window["WebSocket"]) {
alert("Error: Your browser does not support web sockets.")
} else {
socket = new WebSocket("ws://localhost:8080/room");
socket.onclose = function() {
alert("Connection has been closed.");
}
socket.onmessage = function(e) {
messages.append($("<li>").text(e.data));
}
}
});
</script>
The socket = new WebSocket("ws://localhost:8080/room") line is where we open the socket and add event handlers for two key events: onclose and onmessage. When the socket receives a message, we use jQuery to append the message to the list element and thus present it to the user.
Submitting the HTML form triggers a call to socket.send, which is how we send messages to the server.
Build and run the program again to ensure the templates recompile so these changes are represented.
Navigate to http://localhost:8080/ in two separate browsers (or two tabs of the same browser) and play with the application. You will notice that messages sent from one client appear instantly in the other clients:
Currently, we are using templates to deliver static HTML, which is nice because it gives us a clean and simple way to separate the client code from the server code. However, templates are actually much more powerful, and we are going to tweak our application to make some more realistic use of them.
The host address of our application (:8080) is hardcoded at two places at the moment. The first instance is in main.go where we start the web server:
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
The second time it is hardcoded in the JavaScript when we open the socket:
socket = new WebSocket("ws://localhost:8080/room");
Our chat application is pretty stubborn if it insists on only running locally on port 8080, so we are going to use command-line flags to make it configurable and then use the injection capabilities of templates to make sure our JavaScript knows the right host.
Update your main function in main.go:
func main() {
var addr = flag.String("addr", ":8080", "The addr of the application.")
flag.Parse() // parse the flags
r := newRoom()
http.Handle("/", &templateHandler{filename: "chat.html"})
http.Handle("/room", r)
// get the room going
go r.run()
// start the web server
log.Println("Starting web server on", *addr)
if err := http.ListenAndServe(*addr, nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
You will need to import the flag package in order for this code to build. The definition for the addr variable sets up our flag as a string that defaults to :8080 (with a short description of what the value is intended for). We must call flag.Parse() that parses the arguments and extracts the appropriate information. Then, we can reference the value of the host flag by using *addr.
Note
The call to flag.String returns a type of *string, which is to say it returns the address of a string variable where the value of the flag is stored. To get the value itself (and not the address of the value), we must use the pointer indirection operator, *.
We also added a log.Println call to output the address in the terminal so we can be sure that our changes have taken effect.
We are going to modify the templateHandler type we wrote so that it passes the details of the request as data into the template's Execute method. In main.go, update the ServeHTTP function to pass the request r as the data argument to the Execute method:
func (t *templateHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
t.once.Do(func() {
t.templ = template.Must(template.ParseFiles(filepath.Join("templates",
t.filename)))
})
t.templ.Execute(w, r)
}
This tells the template to render itself using data that can be extracted from http.Request, which happens to include the host address that we need.
To use the Host value of http.Request, we can then make use of the special template syntax that allows us to inject data. Update the line where we create our socket in the chat.html file:
socket = new WebSocket("ws://{{.Host}}/room");
The double curly braces represent an annotation and the way we tell our template source to inject data. The {{.Host}} is essentially equivalent of telling it to replace the annotation with the value from request.Host (since we passed the request r object in as data).
Tip
We have only scratched the surface of the power of the templates built into Go's standard library. The text/template package documentation is a great place to learn more about what you can achieve. You can find more about it at http://golang.org/pkg/text/template.
Rebuild and run the chat program again, but this time notice that the chatting operations no longer produce an error, whichever host we specify:
go build -o chat ./chat -addr=":3000"
View the source of the page in the browser and notice that {{.Host}} has been replaced with the actual host of the application. Valid hosts aren't just port numbers; you can also specify the IP addresses or other hostnames provided they are allowed in your environment, for example, -addr="192.168.0.1:3000".
The only way we will know that our application is working is by opening two or more browsers and using our UI to send messages. In other words, we are manually testing our code. This is fine for experimental projects such as our chat application or small projects that aren't expected to grow, but if our code is to have a longer life or be worked on by more than one person, manual testing of this kind becomes a liability. We are not going to tackle Test-driven Development (TDD) for our chat program, but we should explore another useful debugging technique called tracing.
Tracing is a practice by which we log or print key steps in the flow of a program to make what is going on under the covers visible. In the previous section, we added a log.Println call to output the address that the chat program was binding to. In this section, we are going to formalize this and write our own complete tracing package.
We are going to explore TDD practices when writing our tracing code because TDD is a perfect example of a package that we are likely to reuse, add to, share, and hopefully, even open source.
Packages in Go are organized into folders, with one package per folder. It is a build error to have differing package declarations within the same folder because all sibling files are expected to contribute to a single package. Go has no concept of subpackages, which means nested packages (in nested folders) exist only for aesthetic or informational reasons but do not inherit any functionality or visibility from super packages. In our chat application, all of our files contributed to the main package because we wanted to build an executable tool. Our tracing package will never be run directly, so it can and should use a different package name. We will also need to think about the Application Programming Interface (API) of our package, considering how to model a package so that it remains as extensible and flexible as possible for users. This includes the fields, functions, methods, and types that should be exported (visible to the user) and remain hidden for simplicity's sake.
Note
Go uses capitalization of names to denote which items are exported such that names that begin with a capital letter (for example, Tracer) are visible to users of a package, and names that begin with a lowercase letter (for example, templateHandler) are hidden or private.
Create a new folder called trace, which will be the name of our tracing package, alongside the chat folder so that the folder structure now looks like this:
/chat client.go main.go room.go /trace
Before we jump into code, let's agree on some design goals for our package by which we can measure success:
The package should be easy to use
Unit tests should cover the functionality
Users should have the flexibility to replace the tracer with their own implementation
Interfaces in Go are an extremely powerful language feature that allows us to define an API without being strict or specific on the implementation details. Wherever possible, describing the basic building blocks of your packages using interfaces usually ends up paying dividends down the road, and this is where we will start for our tracing package.
Create a new file called tracer.go inside the trace folder and write the following code:
package trace
// Tracer is the interface that describes an object capable of
// tracing events throughout code.
type Tracer interface {
Trace(...interface{})
}
The first thing to notice is that we have defined our package as trace.
Note
While it is a good practice to have the folder name match the package name, Go tools do not enforce it, which means you are free to name them differently if it makes sense. Remember, when people import your package, they will type the name of the folder, and if suddenly a package with a different name is imported, it could get confusing.
Our Tracer type (the capital T means we intend this to be a publicly visible type) is an interface that describes a single method called Trace. The ...interface{} argument type states that our Trace method will accept zero or more arguments of any type. You might think that this is a redundant provision as the method should just take a single string (we want to just trace out some string of characters, don't we?). However, consider functions such as fmt.Sprint and log.Fatal, both of which follow a pattern littered throughout Go's standard library that provides a helpful shortcut when trying to communicate multiple things in one go. Wherever possible, we should follow such patterns and practices because we want our own APIs to be familiar and clear to the Go community.
We promised ourselves that we would follow test-driven practices, but interfaces are simply definitions that do not provide any implementation and so cannot be directly tested. But we are about to write a real implementation of a Tracer method, and we will indeed write the tests first.
Create a new file called tracer_test.go in the trace folder and insert the following scaffold code:
package trace
import (
"testing"
)
func TestNew(t *testing.T) {
t.Error("We haven't written our test yet")
}
Testing was built into the Go tool chain from the very beginning, making writing automatable tests a first-class citizen. The test code lives alongside the production code in files suffixed with _test.go. The Go tools will treat any function that starts with Test (taking a single *testing.T argument) as a unit test, and it will be executed when we run our tests. To run them for this package, navigate to the trace folder in a terminal and do the following:
go test
You will see that our tests fail because of our call to t.Error in the body of our TestNew function:
--- FAIL: TestNew (0.00 seconds) tracer_test.go:8: We haven't written our test yet FAIL exit status 1 FAIL trace 0.011s
Tip
Clearing the terminal before each test run is a great way to make sure you aren't confusing previous runs with the most recent one. On Windows, you can use the cls command; on Unix machines, the clear command does the same thing.
Obviously, we haven't properly written our test and we don't expect it to pass yet, so let's update the TestNew function:
func TestNew(t *testing.T) {
var buf bytes.Buffer
tracer := New(&buf)
if tracer == nil {
t.Error("Return from New should not be nil")
} else {
tracer.Trace("Hello trace package.")
if buf.String() != "Hello trace package.\n" {
t.Errorf("Trace should not write '%s'.", buf.String())
}
}
}
Most packages throughout the book are available from the Go standard library, so you can add an import statement for the appropriate package in order to access the package. Others are external, and that's when you need to use go get to download them before they can be imported. For this case, you'll need to add import "bytes" to the top of the file.
We have started designing our API by becoming the first user of it. We want to be able to capture the output of our tracer in a bytes.Buffer variable so that we can then ensure that the string in the buffer matches the expected value. If it does not, a call to t.Errorf will fail the test. Before that, we check to make sure the return from a made-up New function is not nil; again, if it is, the test will fail because of the call to t.Error.
Running go test now actually produces an error; it complains that there is no New function. We haven't made a mistake here; we are following a practice known as red-green testing. Red-green testing proposes that we first write a unit test, see it fail (or produce an error), write the minimum amount of code possible to make that test pass, and rinse and repeat it again. The key point here being that we want to make sure the code we add is actually doing something as well as ensuring that the test code we write is testing something meaningful.
Consider a meaningless test for a minute:
if true == true {
t.Error("True should be true")
}
It is logically impossible for true to not be true (if true ever equals false, it's time to get a new computer), and so our test is pointless. If a test or claim cannot fail, there is no value whatsoever to be found in it.
Replacing true with a variable that you expect to be set to true under certain conditions would mean that such a test can indeed fail (like when the code being tested is misbehaving) at this point, you have a meaningful test that is worth contributing to the code base.
You can treat the output of go test like a to-do list, solving only one problem at a time. Right now, the complaint about the missing New function is all we will address. In the trace.go file, let's add the minimum amount of code possible to progress with things; add the following snippet underneath the interface type definition:
func New() {}
Running go test now shows us that things have indeed progressed, albeit not very far. We now have two errors:
./tracer_test.go:11: too many arguments in call to New ./tracer_test.go:11: New(&buf) used as value
The first error tells us that we are passing arguments to our New function, but the New function doesn't accept any. The second error says that we are using the return of the New function as a value, but that the New function doesn't return anything. You might have seen this coming, and indeed as you gain more experience writing test-driven code, you will most likely jump over such trivial details. However, to properly illustrate the method, we are going to be pedantic for a while. Let's address the first error by updating our New function to take in the expected argument:
func New(w io.Writer) {}
We are taking an argument that satisfies the io.Writer interface, which means that the specified object must have a suitable Write method.
Note
Using existing interfaces, especially ones found in the Go standard library, is an extremely powerful and often necessary way to ensure that your code is as flexible and elegant as possible.
Accepting io.Writer means that the user can decide where the tracing output will be written. This output could be the standard output, a file, network socket, bytes.Buffer as in our test case, or even some custom-made object, provided it can act like an io.Writer interface.
Running go test again shows us that we have resolved the first error and we only need add a return type in order to progress past our second error:
func New(w io.Writer) Tracer {}
We are stating that our New function will return a Tracer, but we do not return anything, which go test happily complains about:
./tracer.go:13: missing return at end of function
Fixing this is easy; we can just return nil from the New function:
func New(w io.Writer) Tracer {
return nil
}
Of course, our test code has asserted that the return should not be nil, so go test now gives us a failure message:
tracer_test.go:14: Return from New should not be nil
You can see how this hyper-strict adherence to the red-green principle can get a little tedious, but it is vital that we do not jump too far ahead. If we were to write a lot of implementation code in one go, we will very likely have code that is not covered by a unit test.
The ever-thoughtful core team has even solved this problem for us by providing code coverage statistics. The following command provides code statistics:
go test -cover
Provided that all tests pass, adding the -cover flag will tell us how much of our code was touched during the execution of the tests. Obviously, the closer we get to 100 percent the better.
To satisfy this test, we need something that we can properly return from the New method because Tracer is only an interface and we have to return something real. Let's add an implementation of a tracer to our tracer.go file:
type tracer struct {
out io.Writer
}
func (t *tracer) Trace(a ...interface{}) {}
Our implementation is extremely simple: the tracer type has an io.Writer field called out which is where we will write the trace output to. And the Trace method exactly matches the method required by the Tracer interface, although it doesn't do anything yet.
Now we can finally fix the New method:
func New(w io.Writer) Tracer {
return &tracer{out: w}
}
Running go test again shows us that our expectation was not met because nothing was written during our call to Trace:
tracer_test.go:18: Trace should not write ''.
Let's update our Trace method to write the blended arguments to the specified io.Writer field:
func (t *tracer) Trace(a ...interface{}) {
fmt.Fprint(t.out, a...)
fmt.Fprintln(t.out)
}
When the Trace method is called, we use fmt.Fprint (and fmt.Fprintln) to format and write the trace details to the out writer.
Have we finally satisfied our test?
go test -cover PASS coverage: 100.0% of statements ok trace 0.011s
Congratulations! We have successfully passed our test and have 100 percent test coverage. Once we have finished our glass of champagne, we can take a minute to consider something very interesting about our implementation.
The tracer struct type we wrote is unexported because it begins with a lowercase t, so how is it that we are able to return it from the exported New function? After all, doesn't the user receive the returned object? This is perfectly acceptable and valid Go code; the user will only ever see an object that satisfies the Tracer interface and will never even know about our private tracer type. Since they only interact with the interface anyway, it wouldn't matter if our tracer implementation exposed other methods or fields; they would never be seen. This allows us to keep the public API of our package clean and simple.
This hidden implementation technique is used throughout the Go standard library; for example, the ioutil.NopCloser method is a function that turns a normal io.Reader interface into io.ReadCloser where the Close method does nothing (used for when io.Reader objects that don't need to be closed are passed into functions that require io.ReadCloser types). The method returns io.ReadCloser as far as the user is concerned, but under the hood, there is a secret nopCloser type hiding the implementation details.
Note
To see this for yourself, browse the Go standard library source code at
http://golang.org/src/pkg/io/ioutil/ioutil.go and search for the nopCloser struct.
Now that we have completed the first version of our trace package, we can use it in our chat application in order to better understand what is going on when users send messages through the user interface.
In room.go, let's import our new package and make some calls to the Trace method. The path to the trace package we just wrote will depend on your GOPATH environment variable because the import path is relative to the $GOPATH/src folder. So if you create your trace package in $GOPATH/src/mycode/trace, then you would need to import mycode/trace.
Update the room type and the run() method like this:
type room struct {
// forward is a channel that holds incoming messages
// that should be forwarded to the other clients.
forward chan []byte
// join is a channel for clients wishing to join the room.
join chan *client
// leave is a channel for clients wishing to leave the room.
leave chan *client
// clients holds all current clients in this room.
clients map[*client]bool
// tracer will receive trace information of activity
// in the room.
tracer trace.Tracer
}
func (r *room) run() {
for {
select {
case client := <-r.join:
// joining
r.clients[client] = true
r.tracer.Trace("New client joined")
case client := <-r.leave:
// leaving
delete(r.clients, client)
close(client.send)
r.tracer.Trace("Client left")
case msg := <-r.forward:
r.tracer.Trace("Message received: ", string(msg))
// forward message to all clients
for client := range r.clients {
client.send <- msg
r.tracer.Trace(" -- sent to client")
}
}
}
}
We added a trace.Tracer field to our room type and then made periodic calls to the Trace method peppered throughout the code. If we run our program and try to send messages, you'll notice that the application panics because the tracer field is nil. We can remedy this for now by making sure we create and assign an appropriate object when we create our room type. Update the main.go file to do this:
r := newRoom() r.tracer = trace.New(os.Stdout)
We are using our New method to create an object that will send the output to the os.Stdout standard output pipe (this is a technical way of saying we want it to print the output to our terminal).
Rebuild and run the program and use two browsers to play with the application, and notice that the terminal now has some interesting trace information for us:
Now we are able to use the debug information to get an insight into what the application is doing, which will assist us when developing and supporting our project.
Once the application is released, the sort of tracing information we are generating will be pretty useless if it's just printed out to some terminal somewhere, or even worse, if it creates a lot of noise for our system administrators. Also, remember that when we don't set a tracer for our room type, our code panics, which isn't a very user-friendly situation. To resolve these two issues, we are going to enhance our trace package with a trace.Off() method that will return an object that satisfies the Tracer interface but will not do anything when the Trace method is called.
Let's add a test that calls the Off function to get a silent tracer before making a call to Trace to ensure the code doesn't panic. Since the tracing won't happen, that's all we can do in our test code. Add the following test function to the tracer_test.go file:
func TestOff(t *testing.T) {
var silentTracer Tracer = Off()
silentTracer.Trace("something")
}
To make it pass, add the following code to the tracer.go file:
type nilTracer struct{}
func (t *nilTracer) Trace(a ...interface{}) {}
// Off creates a Tracer that will ignore calls to Trace.
func Off() Tracer {
return &nilTracer{}
}
Our nilTracer struct has defined a Trace method that does nothing, and a call to the Off() method will create a new nilTracer struct and return it. Notice that our nilTracer struct differs from our tracer struct in that it doesn't take an io.Writer interface; it doesn't need one because it isn't going to write anything.
Now let's solve our second problem by updating our newRoom method in the room.go file:
func newRoom() *room {
return &room{
forward: make(chan []byte),
join: make(chan *client),
leave: make(chan *client),
clients: make(map[*client]bool),
tracer: trace.Off(),
}
}
By default, our room type will be created with a nilTracer struct and any calls to Trace will just be ignored. You can try this out by removing the r.tracer = trace.New(os.Stdout) line from the main.go file: notice that nothing gets written to the terminal when you use the application and there is no panic.
A quick glance at the API (in this context, the exposed variables, methods, and types) for our trace package highlights that a simple and obvious design has emerged:
The
New()- method-creates a new instance of a TracerThe
Off()- method-gets a Tracer that does nothingThe
Tracerinterface - describes the methods Tracer objects will implement
I would be very confident to give this package to a Go programmer without any documentation or guidelines, and I'm pretty sure they would know what do to with it.
Note
In Go, adding documentation is as simple as adding comments to the line before each item. The blog post on the subject is a worthwhile read (http://blog.golang.org/godoc-documenting-go-code), where you can see a copy of the hosted source code for tracer.go that is an example of how you might annotate the trace package. For more information, refer to https://github.com/matryer/goblueprints/blob/master/chapter1/trace/tracer.go.
In this chapter, we developed a complete concurrent chat application and our own simple package to trace the flow of our programs to help us better understand what is going on under the hood.
We used the net/http package to quickly build what turned out to be a very powerful concurrent HTTP web server. In one particular case, we then upgraded the connection to open a web socket between the client and server. This means that we can easily and quickly communicate messages to the user's web browser without having to write messy polling code. We explored how templates are useful to separate the code from the content as well as to allow us to inject data into our template source, which let us make the host address configurable. Command-line flags helped us give simple configuration control to the people hosting our application while also letting us specify sensible defaults.
Our chat application made use of Go's powerful concurrency capabilities that allowed us to write clear threaded code in just a few lines of idiomatic Go. By controlling the coming and going of clients through channels, we were able to set synchronization points in our code that prevented us from corrupting memory by attempting to modify the same objects at the same time.
We learned how interfaces such as http.Handler and our own trace.Tracer interface allow us to provide disparate implementations without having to touch the code that makes use of them, and in some cases, without having to expose even the name of the implementation to our users. We saw how just by adding a ServeHTTP method to our room type, we turned our custom room concept into a valid HTTP handler object, which managed our web socket connections.
We aren't actually very far away from being able to properly release our application, except for one major oversight: you cannot see who sent each message. We have no concept of users or even usernames, and for a real chat application, this is not acceptable.
In the next chapter, we will add the names of the people responding to their messages in order to make them feel like they are having a real conversation with other humans.
