Security and privacy, as practical matters, have continued to gain interest, especially in the technology industry. The cybersecurity market is booming and continuing to grow. The industry moves fast with innovations and research coming out constantly. Not only has the interest and speed of security picked up, but the scale of applications along with the risk have also grown by orders of magnitude. The industry needs a programming language that is simple to learn, cross-platform, and efficient on a large scale. Go is the perfect fit, having an extremely powerful standard library, short learning curve, and blazing speed.

In this chapter, we will cover the following topics:

• Go's history, language design, criticisms, community, and learning tips
• Why use Go for security
• Setting up a development environment and writing your first program
• Running the example programs

Go is an open source programming language that was created by Google and distributed under a BSD-style license. A BSD license allows anyone to use Go free of charge, as long as the copyright notice is retained and the Google name is not used for endorsement or promotion. Go is heavily influenced by C, but has simpler syntax, and better memory safety and garbage collection. Sometimes, Go is described as a modern-day C++. I think that is too much of a simplification, but Go is definitely a simple but modern language.

The original goal of Go was to create a new language that is simple, reliable, and efficient. As mentioned, Go is heavily influenced by C programming language. The language itself is very simple, with only 25 keywords. It was built to integrate well with IDEs, but not to be dependent on them. In my experience, anyone who has tried Go has found it very user-friendly with a short learning curve.

One of the main goals of Go was to deal with some of the negative aspects of C++ and Java code, while retaining the performance. The language needed to be simple and consistent to manage very large development teams.

Variables are statically typed, and applications are compiled quickly to statically linked binaries. Having a single statically linked binary makes it very easy to create lightweight containers. The final applications run fast as well, running close to C++ and Java performance and much faster than interpreted languages such as Python. There are pointers, but there is no pointer arithmetic allowed. Go does not tout itself as an object-oriented programming language, and it does not formally have classes in the traditional sense; however, it does contain a number of mechanisms that closely resemble an object-oriented programming language. This is discussed in more depth in the following chapter. Interfaces are used heavily, and composition is the equivalent of inheritance.

Go has many interesting features. One feature that stands out is the built-in concurrency. Just put the word "go" before any function call, and it will spawn a lightweight thread to execute the function. Another feature that is quite important is the dependency management, which is very efficient. The dependency management is part of the reason Go compiles incredibly fast. It does not re-include the same header files multiple times, the way C++ does. Go also has built-in memory safety, and a garbage collector handles clean-up of unused memory. The standard library in Go is pretty impressive too. It is modern and contains networking, HTTP, TLS, XML, JSON, database, image manipulation, and cryptography packages. Go also supports Unicode, allowing all kinds of characters to be used in source code.

The Go toolchain is central to the ecosystem. It provides tools to download and install remote dependencies, run unit tests and benchmarks, generate code, and format code according to the Go formatting standards. It also includes the compiler, linker, and assembler, which compile very quickly and also allow for easy cross-compiling by simply changing the GOOS and GOARCH environment variables.

Some features were excluded from the Go language. Generics, inheritance, assertions, exceptions, pointer arithmetic, and implicit type conversions were all left out of Go. Many features were omitted intentionally, namely generics, assertions, and pointer arithmetic. The authors left out some features because they wanted to maintain performance, keep the language specification as simple as possible, or they could not agree on the best way to implement, or because a feature was too controversial. Inheritance was also left out intentionally in favor of using interfaces and composition instead. Some other features, such as generics, were left out because there was too much debate concerning their proper implementation, but they may show up in Go 2.0. The authors recognized that it is much easier to add a feature to a language than to take one away.

Go is a relatively young language, with its inception in 2007 and open sourcing in 2009. It started at Google as a 20% project with Robert Griesemer, Rob Pike, and Ken Thompson. A 20% project means that the project's developers spent 20% of their time working on it as an experimental side project. Go 1.0 was officially released in March 2012. It was planned from the beginning to be an open source language. Until Go 1.5, the compiler, linker, and assembler were written in C. After version 1.5, everything was written in Go.

Google initially launched Go for Linux and macOS, and the community drove the effort for other platforms, namely Windows, FreeBSD, OpenBSD, NetBSD, and Solaris. It has even been ported to IBM z Systems mainframes. Bill O'Farrell of IBM gave a talk at GopherCon 2016 in Denver titled Porting Go to the IBM z Architecture (https://www.youtube.com/watch?v=z0f4Wgi94eo).

Google is known to use Python, Java, and C++. It is understandable why they chose those languages too. They each fill certain roles and have their own strengths and weaknesses. Go was an attempt to create a new language that fits the needs of Google. They needed software that could perform extremely well under heavy loads, support concurrency, and that was easy to read and write, and compile quickly.

The triggering event for starting the Go project was dealing with a massive C++ code base that took hours to build because of the way C++ handles dependencies and re-includes headers (https://www.youtube.com/watch?v=bj9T2c2Xk_s (37:15)). This is why one of Go's main goals was to compile quickly. Go helped turn hours of compile time to seconds because it handles dependencies much more efficiently than C++.

Discussions for Go 2.0 have begun, but they are still in the conceptual stages. There is no timeline for a release, and there is no rush to release a new major version.

Go is still a young language, but it has seen growing adoption rates and has continued to increase in popularity. Go was the TIOBE Language of the year in 2009 and 2016:

One of the expectations expressed by the Go team was the anticipation that Go would draw a lot of C/C++ and Java developers, but they were surprised when a large number of the users came from scripting languages such as Python and Ruby. Others, like myself, found Go to be a natural complement to Python, a great language. However, which language do you go to when you need something much more powerful? Some large companies have demonstrated that Go is stable for large-scale production use, including Google, Dropbox, Netflix, Uber, and SoundCloud.

The first Go conference, named GopherCon, was held in 2014. Since then, GopherCon has been held every year. Read more about GopherCon at https://gophercon.com. I had the privilege of speaking at the 2016 GopherCon about packet capturing and had a great experience (https://www.youtube.com/watch?v=APDnbmTKjgM).

There are a few criticisms that show up repeatedly in the community. Probably the most notorious and most-discussed criticism is the lack of generics. This leads to repeated code to handle different data types. Interfaces can be used to some extent to alleviate this problem. We might see generics in a future release, as the authors have shown openness to generics, but they did not rush through an important design decision.

The next criticism often heard is the lack of exceptions. The developer must explicitly handle or ignore each error. Personally, I found this to be a refreshing change. It's not really that much more work, and you have full control over the code flow. Sometimes with exceptions you are not positive where it will get caught as it bubbles up. With Go, you can easily follow the error-handling code.

Go has a garbage collector that handles memory cleanup. The garbage collector has been upgraded over time and continues to improve. The garbage collector does have a minor performance impact, but saves the developer a lot of thinking and worrying. Go was originally described as a systems programming language, and the lack of control over the memory was restrictive for very low-level applications. Since then, they have pivoted away from calling Go a systems programming language. If you need low-level control over memory, then you will have to write portions of code in C.

The go executable is the main application of the Go toolchain. You can pass a command to go, and it will take the appropriate action. The toolchain has tools to run, compile, format source code, download dependencies, and more. Let's look at the full list, which is obtained as an output from the go help command or just go by itself:

• build: This compiles packages and dependencies
• clean: This removes object files
• doc: This shows documentation for a package or symbol
• env: This prints Go environment information
• generate: This is the code generator
• fix: This upgrades Go code when a new version is released
• fmt: This runs gofmt on package sources
• get: This downloads and installs packages and dependencies
• help: This provides more help on a specific topic
• install: This compiles and installs packages and dependencies
• list: This lists packages
• run: This compiles and runs Go programs
• test: This runs unit tests and benchmarks
• vet: This examines source code for bugs
• version: This shows the Go version

More information about these commands is available at https://golang.org/cmd/.

Everyone knows that the best swords have names, and the best programming languages have mascots. Go's mascot is the gopher. The gopher has no name. It has a jelly bean shaped body, microscopic limbs, gigantic eyes, and two teeth. It was designed by Renee French, and its copyright comes under the Creative Commons Attribution 3.0 license. This means that you can play with the images, but you must give credit to their creator, Renee French, wherever they are used.

Renee French gave a talk at GopherCon 2016 in Denver, entitled The Go Gopher: A Character Study, explaining how the gopher came to be, the various mediums and forms it has taken, and the tips on drawing it in various situations (https://www.youtube.com/watch?v=4rw_B4yY69k).

You can generate a custom gopher avatar at https://gopherize.me/ and read more about the Go gopher at https://blog.golang.org/gopher.

If you have never used Go before, have no fear. It has a gentle learning curve and is simple enough to learn in just a day or two. The best place to start is https://tour.golang.org/. This is a basic tour of the Go programming language. If you have already gone through the tour, then you should already have the foundation to make it through this book just fine. If you are working through this book and have not taken the tour, you may come across a concept you are not familiar with that is not explained here. The tour is a good place to learn and practice.

Since there are only 25 reserved keywords in the language specification, it is short enough to be understood by "mortals". You can read more about the specs at https://golang.org/ref/spec.

You must be already be familiar with most of these keywords. These are: if, else, goto, for, import, return, var, continue, break, range, type, func, interface, package, const, map, struct, select, case, switch, go, defer, chan, fallthrough, and default.

The tour will help you learn the keywords, syntaxes, and basics of the data structures. The playground in the tour lets you practice writing and running code in the browser.

There are several aspects that appeal to me about Go. Concurrency, speed, and simplicity are the most important things to me. The language is very simple and easy to learn. There are no try, catch, and exception flows. Though several people cite the tedious error handling as a criticism, I find it refreshing to have a simple language that does not hide a lot of magic behind the scenes and does exactly what it says. The go fmt tool standardizes formatting, making it easy to read code from others, and eliminates the burden of defining your own standard.

Go provides a feeling of scalability and reliability and is actually an enjoyable experience. Before Go, the primary option for fast, compiled code was C++, and it was no simple task to manage the header files and build processes for different platforms. C++ has become a very complicated language over the years and is not nearly as approachable as Go for most people.

I think we all understand that there is no such thing as the best programming language, but there are different tools for different jobs. Go excels in performance and concurrency. Some of its other benefits include the ability to compile down to a single executable and cross-compile easily. It also has a modern standard library well-suited for networked applications.

The ease of cross-compiling makes for some interesting use cases in the security field. Here are a couple of use cases for cross-compiling in security:

• Penetration testers can use a Raspberry Pi to compile custom Go reverse shells for Windows, macOS, and Linux, and attempt to deploy them.
• Network defenders can have one central database to store all honeypot information provided from honeypot servers, and then cross-compile the honeypot servers. This would allow them to easily deploy a consistent application across all platforms, including Windows, mac, and Linux.
• Network defenders could deploy incredibly lightweight honeypots throughout their network in the form of a Docker container with a single statically linked binary. Containers would be quick to create and destroy, using minimal bandwidth and server resources.

When you ask yourself whether Go is a good language choice, it may help to compare Go with some of the other top language choices.

Python is a popular language in the security field. This is most likely because of its ubiquity, short learning curve, and plethora of libraries. There are already several useful tools for security written in Python, namely Scapy for packet capturing, Scrapy for web scraping, Immunity for debugging, Beautiful Soup for parsing HTML, and Volatility for memory forensics. Many vendors and service providers also provide API examples in Python.

Python is easy to learn, and there are plenty of resources. Go is also easy to write and has a gentle learning curve. The learning curve and the ease of programming is not a major differentiating factor between Go and Python in my opinion. This biggest distinction, and where Python falls short, is performance. Python cannot compete with Go in terms of performance. Part of it is the interpreted nature of Python, but a larger factor is the global interpreter lock or GIL. The GIL prevents the interpreter from using more than one CPU worth of processing power, even with multiple threads executing. There are some ways to get around this, such as using multiprocessing, but this has its own drawbacks and limitations, as it actually forks a new process. Other options are using Jython (Python on Java) or IronPython (Python on .NET), and these have no GIL.

One of Java's greatest strengths is the ability to write once, run anywhere (WORA). This is incredibly valuable if you have to do anything involving GUI, graphics, or audio. Go certainly does not beat Java in its ability to create GUIs, but it is cross-platform and supports cross-compiling.

Java is mature and widely adopted with lots of resources available. There are more options with Java libraries than Go packages. Java is the more verbose of the two languages. The Java ecosystem is more complex with several options for build tools and package managers. Go is much simpler and more standardized. These differences could simply be attributed to the age difference between the languages, but it may still affect your language choice.

In certain situations, the Java virtual machine (JVM) can be too resource intensive in terms of memory or startup time. If you need to pipe together several command-line Java applications, the startup time for the JVM just to run a series of short-lived programs can be a significant performance hit. In terms of memory, if you need to run several instances of the same application, then the memory required to run each JVM can add up. The JVM can also be restricting since it creates a sandbox and limits your access to the host machine. Go compiles down to native machine code and thus has no need for a virtual machine layer.

Go is well-documented and the community continues to grow and provide more resources. It is an easy language to learn, especially for experienced programmers. Concurrency is a bit simpler and built into the language, as opposed to a library package.

C++ does offer a little more control since the developer is in charge of memory management and there is no garbage collector. For this same reason, C++ will have slightly better performance. In some cases, Go can actually outperform C++.

C++ is very mature and has a large set of third-party libraries. Libraries are not always cross-platform and can have complex makefiles. Cross-compiling is much simpler in Go and can be done with the Go toolchain.

Go compiles with more efficiency because it has better dependency management. C++ can re-include the same header file multiple times and cause compilation times to balloon. The package system is more consistent and standardized in Go. Threads and concurrency are native to Go and require platform-specific libraries in C++.

The maturity of C++ has also led to the language growing and becoming more complicated over time. Go is a refreshing change with a simple-yet-modern language. C++ is not as friendly to beginners as Go.

All of the examples in this book will run across major platforms, Windows, macOS, and Linux. Having said that, the examples were primarily written and developed on Ubuntu Linux, and this is the recommended platform for the following examples.

Ubuntu Linux is available for free at https://www.ubuntu.com/download/desktop. The download page may ask for a donation, but you can choose to download for free. Ubuntu is not required, but the book will be easier to follow if you have the same environment. Other Linux distributions should work equally well, but I strongly recommend that you use a Debian-based distribution. Most of the Go code examples in this book will work on Windows, Linux, and Mac without any modification. Certain examples may be Linux- and Mac-specific, such as file permissions, which are not treated similarly in Windows. Any example that is specific to a platform is mentioned.

You can install Ubuntu for free inside a virtual machine or as your primary operating system. As long as your system has enough CPU, RAM, and disk space, I recommend that you use a virtual machine with Oracle VirtualBox, which is available at https://www.virtualbox.org/. VMWare Player is an alternative to VirtualBox and is available at https://www.vmware.com/products/player/playerpro-evaluation.html.

Download and install VirtualBox, and then, download the Ubuntu desktop ISO file. Create a virtual machine, have it boot the Ubuntu ISO, and choose the Install option. Once you have installed Ubuntu and logged in as your user, you can install the Go programming language. Ubuntu makes this incredibly easy by providing a package. Just open a Terminal window and run the following command:

sudo apt-get install golang-go

Using sudo elevates your privileges in order to install and may ask you for your password. If everything was successful, you will now have access to the go executable, which contains the whole toolchain. You can run go help or go by itself for usage instructions.

If you are not using Ubuntu or want to install the latest version, you can download the latest version from https://golang.org/dl. The Windows and Mac installer will take care of updating your PATH environment variable, but in Linux you will have to move the extracted contents to a desired location, such as /opt/go, and then update your PATH environment variable manually to include the location. Consider this example:

# Extract the downloaded Go tar.gz
tar xzf go1.9.linux-amd64.tar.gz
# Move the extracted directory to /opt
sudo mv go /opt
# Update PATH environment variable to include Go's binaries
echo "export PATH=$PATH:/opt/go/bin" >> ~/.bashrc

Now restart your Terminal for the changes to take effect. If you are using a shell other than Bash, you will need to update the proper RC file for your shell.

If you are not using Ubuntu, you can still install Go easily. The Go website provides multiple installation formats on the Downloads page at https://golang.org/dl/.

The first option is to use the package manager for the Linux distribution to install Go. Most major distributions have a package for Go. Names vary, so a web search may be necessary to get the exact package name. If there is no package available, you can simply download the precompiled Linux tarball and extract it. A good place to extract the contents is /opt/go. Then, add /opt/go/bin to your PATH environment variable the same way as described in the previous section.

An official Windows installer is available, which makes installation as simple as running the installer. You may need to modify the environment variables and update your %PATH% variable. In Windows 10, this can be found by navigating to Control Panel | System | Advanced System Settings | Environment Variables.

An official installer is also available for Mac. After running the installer, Go will be available in your PATH variable.

At this point, your environment should have Go installed and you should be able to run go executable from your Terminal window. The go program is how you access the Go toolchain. You can test it by running this command:

go help

Now we are ready to write a first Hello World program to ensure that our environment is fully functional. Before we start coding, though, we need to create a proper workspace.

Go has a standard folder structure for a workspace. It is important to conform to certain standards for the Go toolchain to work properly. You can create a workspace directory anywhere you want and name it anything you like. For the lab environment, we will simply use the Home directory as the Go workspace. This means that source files will reside in ~/src, packages will be built in ~/pkg, and executables will be installed to ~/bin.

In order for most of the Go toolchain to work, the GOPATH environment variable must be set. The GOPATH specifies what directory you treat as your workspace. The GOPATH environment variable must be set before you can build packages. For more help and information, call the go help command in the Terminal by running this command:

go help gopath

We need to tell Go to treat our home directory as the workspace. This is done by setting the GOPATH environment variable. You can set GOPATH in three ways:

• The first way is to set it manually each time you run the go command. Consider this example:

      GOPATH=$HOME go build hello

• You can also set the GOPATH variable so that it stays set until you close your Terminal and the environment variable is lost:

      export GOPATH=$HOME

• The third option is to set the GOPATH environment variable permanently as follows:

1. 1. Add it to your shell startup script, .bashrc. This will set the variable every time you start the Terminal.
   2. Run this to ensure that GOPATH is set whenever you open future Terminal/shell sessions:

              echo "export GOPATH=$HOME" >> $HOME/.bashrc

1. 3. Restart your Terminal for the changes to take effect. If you are using Zsh or an alternative shell, you will need to update the respective RC file.

We're about to write our first program in our new hello directory. You will first need to choose which editor to use. Fortunately, working with Go does not require any special IDE or editor. The Go toolchain integrates easily into many editors and IDEs. Your options range from using a simple text editor, such as Notepad, to full-fledged IDEs dedicated to Go.

I recommend that you start with a simple text editor, such as nano or gedit, since these are included with Ubuntu, easy to use, and support syntax highlighting for Go out of the box. Feel free to choose another editor or IDE though.

Plugins exist for many text editors and IDEs to add Go support. For example, Visual Studio Code, Emacs, Sublime Text, JetBrains IntelliJ, Vim, Atom, NetBeans, and Eclipse all have Go plugins. There are a couple of Go-specific IDEs, namely JetBrains GoLand and LiteIDE, both of which are cross-platform.

Start with the nano or gedit command and explore other editors and IDEs after you are comfortable with Go. This book will not compare the editors or cover how to configure them.

Within the ~/src directory, any directory you create is a package. The name of your directory becomes the name of the package or application. We need to first make sure that the src directory exists. Tilde (~) is a shortcut for your home directory similar to the $HOME variable. Refer to the following code block:

mkdir ~/src

Let's create a new package named hello for our first application:

cd ~/src
mkdir hello

A package is simply a directory. You can have one or more source files inside a package. Any subdirectories are treated as separate packages. A package can be an application with a main() function (package main), or it can be a library that can only be imported to other packages. This package doesn't have any files yet, but we'll write the first file in a moment. Don't worry too much about package structure for now. You can read more about package paths at https://golang.org/doc/code.html#PackagePaths.

The simplest package you can have is a single file inside a directory. Create a new file, ~/src/hello/hello.go, and put the following code inside:

package main

import "fmt"

func main() {
   fmt.Println("Hello, world.")
}

The simplest way to execute a program is with the go run command. The following command will run the file without leaving behind an executable file:

go run ~/src/hello/hello.go

To compile and build an executable file, use the go build command. When running go build you must pass a path to a package. The package path you provide is relative to $GOPATH/src. Since our package is in ~/src/hello, we would run the command as follows:

go build hello

We can actually call go build from anywhere as long as we have a $GOPATH set. The executable binary that is created will be output in the current working directory. You can then run it with this command:

./hello

The go build tool is good for generating an executable file in your current working directory, but there is a way to build and install your applications so that the executables are all collected in the same location.

When you run go install it puts the output file in a default location of $GOPATH/bin. In our case, we set $GOPATH equal to our $HOME. So the default bin directory would be $HOME/bin.

You can override the location by setting the GOBIN environment variable if you want it to install somewhere else. To install our hello program, we will run the following command:

go install hello

This will build and create an executable file, ~/bin/hello. The bin directory will get created automatically if it does not already exist. If you run the install command multiple times, it will rebuild and overwrite the executable in the bin directory. Then the application can be run with this:

~/bin/hello

You can add ~/bin to your PATH environment variable for convenience. Doing so will allow you to run the applications from any working directory. To add the bin directory to your PATH, run this in the Terminal:

echo "export PATH=$PATH:$HOME/gospace/bin" >> ~/.bashrc

Be sure to restart your Terminal after that to refresh the environment variables. After that you can run the hello application by simply typing the following into the Terminal:

hello

Installing the application is completely optional. You don't have to install programs to run or build them. You can always build and run from your current working directory when developing, but it can be convenient to install finished applications that get used.

The go fmt command is used to format source code files to meet Go formatting standards.

This will make sure that indentation is accurate and there are no excessive blank spaces, among other things. You can format a single Go source code file or a whole package at once. It is good practice to follow Go coding standards and run go fmt on your files so that you will have no doubt that your code follows the guidelines. Read more on formatting at https://golang.org/doc/effective_go.html#formatting.

The examples provided in this book are all self-contained. Every example is a full program and can be run. Most examples are short and demonstrate one specific topic. While the examples can be used as standalone programs, some of them may have limited use. They are intended to be references and used like a cookbook for building your own projects. Because each example is a self-contained main package, you can use the go build command to get an executable and go run to run the file. Here are some more details about the various options for building and running programs.

If you build a file, it will generate an executable named after the Go file. Run the following command:

go build example.go

This will give you an executable named example that could be executed like this:

./example

You don't have to build a file and get an executable if you only want to run it. The go run option allows you to run the .go file without leaving an executable behind. You can still pass in arguments as if it was a regular executable, like this:

go run example.go arg1 arg2

If a program is split into multiple files, you can pass all of them to the build command. For example, if you have a main.go file and an utility.go file containing extra functions, you could build them by running the following command:

go build main.go utility.go

If you tried to build main.go by itself, it would not be able to find the references to the functions in utility.go.

If a package contains multiple Go files that need to be built, it is tedious to pass each file to the build command. If you run go build with no arguments inside a folder, it will attempt to build all the .go files in the directory. If one of those files contains a package main statement at the top, it will generate an executable named after the directory name. If you write a program, it is possible to write a package that contains no main file and is used only as a library to be included in other projects.

Installing a program is similar to building one but, instead of running go build, you run go install. You can run it inside a directory, pass it an absolute directory path, and pass it a directory path relative to the $GOPATH environment variable or on a file directly. Once a program has been installed, it goes into your $GOBIN, which you should have already set. You should have already added $GOBIN to your $PATH as well so that you can run the installed programs directly from your command line no matter what directory you are currently in. Installing is totally optional, but it is convenient for certain programs, especially for the ones you want to save or use frequently.

After reading this chapter, you should have a general understanding of the Go programming language and some of its key features. You should also have a version of Go installed on your machine with your environment variables set up. If you need more instructions on installing and testing your environment, refer to the Go documentation at https://golang.org/doc/install.

In the next chapter, we will look closer at the Go programming language, learning about the design, data types, keywords, features, control structures, and where to get help and find documentation. If you are already familiar with Go, it should be a good review to reinforce your foundational knowledge. If you are new to Go, it will serve as a primer to prepare you for the rest of the book.