An operating system is the kind of software that allows you to communicate with the hardware, which means that you cannot use your hardware without an operating system. Unix is an operating system with many variants that have many things in common including their programming interface.

The Unix operating system was mainly programmed in C and not entirely in the assembly language, which makes it portable to other computer architectures without having to rewrite everything from scratch. It is important to understand that even if you are developing a Go program on a Unix machine, at the end of the day, your code will be translated to C functions and system calls because this is the only way to directly communicate with the Unix kernel. The main benefits you get from writing Go code instead of C code are smaller programs with less silly bugs. You will learn more about this in Chapter 3, Advanced Go Features.

As this book will use Go, you will need to have a version of Go installed on your Unix machine. The good news is that there is a port of the Go programming language for almost all modern Unix systems including macOS, Linux, and FreeBSD. There is also a Windows port of Go, but this book will not deal with Microsoft Windows.

Although there is a good chance that your Unix variant has a package for Go, you can also get Go from https://golang.org/dl/.

In this chapter, you will learn the following topics:

• Systems programming
• The advantages and disadvantages of Go
• The states of a Unix process
• Two Go tools: gofmt and godoc
• The features of the latest Go version (1.8)

This book has three parts. The first part, which includes this chapter, is about Go and the Go features that can be handy when developing systems software: this does not mean that you should use all of them when developing your programs. The second part is all about programming with files, directories, and processes, which is the most common type of systems software. The third part explores goroutines, web applications, and network programming in Go, which is the most advanced type of systems software. The good thing is that you do not need to read the third part of the book right away.

Systems programming is a special area of programming on Unix machines. Note that systems programming is not limited to Unix machines: it is just that this book deals with the Unix operating system only. Most commands that have to do with system administration tasks, such as disk formatting, network interface configuration, module loading, and kernel performance tracking, are implemented using the techniques of systems programming. Additionally, the /etc directory, which can be found on all Unix systems, contains plain text files that deal with the configuration of a Unix machine and its services and are also manipulated using systems software.

You can group the various areas of systems software and related system calls in the following sets:

• File I/O: This area deals with file reading and writing operations, which is the most important task of an operating system. File input and output must be fast and efficient, and above all, reliable.
• Advanced file I/O: Apart from the basic input and output system calls, there are also more advanced ways to read or write to a file including asynchronous I/O and non-blocking I/O.
• System files and configuration: This group of system software includes functions that allow you to handle system files, such as /etc/passwd, and get system specific information, such as system time and DNS configuration.
• Files and directories: This cluster includes functions and system calls that allow the programmer to create and delete directories and get information such as the owner and the permissions of a file or a directory.
• Process control: This group of software allows you to create and interact with Unix processes.
• Threads: When a process has multiple threads, it can perform multiple tasks. However, threads must be created, terminated, and synchronized, which is the purpose of this collection of functions and system calls.
• Server processes: This set includes techniques that allow you to develop server processes, which are processes that get executed in the background without the need for an active terminal. Go is not that good at writing server processes in the traditional Unix way: but let me explain this a little more. Unix servers such as Apache use fork(2) to create one or more child processes (this process is called forking and refers to cloning the parent process into a child process) and continue executing the same executable from the same point, and most importantly, sharing memory. Although Go does not offer an equivalent to the fork(2) function, this is not an issue because you can use goroutines to cover most of the uses of fork(2).
• Interprocess communication: This set of functions allows processes that run on the same Unix machine to communicate with each other using features such as pipes, FIFOs, message queues, semaphores, and shared memory.
• Signal processing: Signals offer processes a way of handling asynchronous events, which can be very handy. Almost all server processes have extra code that allows them to handle Unix signals using the system calls of this group.
• Network programming: This is the art of developing applications that work over computer networks with the help of TCP/IP and is not systems programming per se. However, most TCP/IP servers and clients are dealing with system resources, users, files, and directories. So, most of the time, you cannot create network applications without doing some kind of systems programming.

The challenging thing with systems programming is that you cannot afford to have an incomplete program; you can either have a fully working, secure program that can be used on a production system or nothing at all. This mainly happens because you cannot trust end users and hackers. The key difficulty in systems programming is the fact that an erroneous system call can make your Unix machine misbehave or, even worse, crash!

Most security issues on Unix systems usually come from wrongly implemented systems software because bugs in systems software can compromise the security of an entire system. The worst part is that this can happen many years after using a certain piece of software.

Back when Unix was first introduced, the only way to write systems software was using C; nowadays, you can program systems software using programming languages including Go, which will be the subject of this book.

You should understand that the two main benefits you get from using a programming language other than C for developing systems software are as follows:

• Using a modern programming language along with its tools
• Simplicity, as you usually have to write, debug, and maintain less code

Apart from Go, other good candidates for developing system tools are Python, Perl, Rust, and Ruby.

The only way you can learn systems programming is by developing your own utilities using this book as a reference and a tutorial. At first, you will make a large amount of ridiculous mistakes, but as you get better, you will make a smaller amount of much more clever and hard to debug mistakes! However, it is fine to try new things when learning. In fact, it is necessary to try new things and fail because this means that you are really learning something new. Just make sure that you do not use a production web server for learning systems programming.

If you have difficulties finding out what to develop, you can start by creating your own versions of some of the existing Unix command line utilities such as ls(1), mkdir(1), ln(1), wc(1), and which(1). You do not have to create a fully featured version of each one of them with support for all command-line options; what is important is to develop a stable and secure version that implements the main functionality and works without problems.

Go is a modern generic purpose open source programming language that was officially announced at the end of 2009. It began as an internal Google project and has been inspired by many other programming languages including C, Pascal, Alef, and Oberon. Its spiritual fathers are Robert Griesemer, Ken Thomson, and Rob Pike, who designed Go as a language for professional programmers who want to build reliable and robust software. Apart from its syntax and standard functions, Go comes with a pretty rich standard library.

At the time of writing this book, the latest stable Go version is 1.8, which includes some handy new features including the following: feel free to skip this if you have not used Go before:

• New conversion rules exist that allow you to easily convert between types that are almost equal provided that some criteria are met. You can fix the import paths of the golang.org/x/net/name form to just the name of the Go source file using the go tool command without having to open the source files yourselves.
• The operation of the tool is stricter in some cases and looser in cases that used to generate false positives.
• There is now a default value for GOPATH Environment Variables when GOPATH is undefined. For Unix systems, the default value is $HOME/go.
• There are various improvements to the Go runtime that speed up Go.
• There is a sort.slice() function that allows you to sort a slice by providing a comparator callback instead of implementing sort.Interface.
• There is now a Shutdown method to http.Server.
• There exist various small changes to the database/sql package that give the developer more control over queries.
• You can create bugs using the go bug command.

You can easily find your version of Go using this command:

$ go version
go version go1.7.5 darwin/amd64  

The previous output is from a macOS machine hence the darwin string. A Linux machine would give the following kind of output:

$ go version
go version go1.3.3 linux/amd64

You will learn more about go tool, which you will use all the time, in the next chapters.

As I can imagine, you must be impatient to see some Go code; so here is the Go version of the famous Hello World program:

package main 
 
import "fmt" 
 
// This is a demonstrative comment! 
func main() { 
   fmt.Println("Hello World!") 
} 

If you are familiar with C or C++, you will find Go code pretty easy to understand. Each file that contains Go code begins with a package declaration followed by the needed import declarations. The package declaration shows the package that this file belongs to. Note that semicolons are not required for successfully terminating a Go statement unless you want to put two or more Go statements in the same line.

In Chapter 2, Writing Programs in Go, you will find out how to compile and execute Go code. For now, it is enough to remember that Go source files are stored using the .go file extension: your task is to choose a descriptive filename.

The Go distribution comes with a plethora of tools that can make your life as a programmer easier. The two most useful of them are gofmt and godoc.

The gofmt utility formats Go programs in a given way, which is really important when different people are going to work with the same code for a big project. You can find more information about gofmt at https://golang.org/cmd/gofmt/.

The following is a poorly formatted version of the hw.go program that is hard to read and understand:

$ cat unglyHW.go
package main
    
import
    "fmt"
    
// This is a demonstrative comment!
        func main() {
  fmt.Println("Hello World!")
    
}

Processing the previous code, which is saved as unglyHW.go with gofmt, generates the following easy to read and comprehend output:

$ gofmt unglyHW.go
package main
    
import "fmt"
    
// This is a demonstrative comment!
func main() {
      fmt.Println("Hello World!")
    
}

Remembering that the gofmt utility does not automatically save the generated output is important, which means that you should either use the -w option followed by a valid filename or redirect the output of gofmt to a new file.

The godoc utility allows you to see the documentation of existing Go packages and functions. You can find more information about godoc at http://godoc.org/golang.org/x/tools/cmd/godoc.

The following screenshot shows the output of the godoc command generated on a Terminal when asked for information about the Println() function of the fmt package:

Another handy feature of godoc is that it can start its own web server and allow you to see its documentation using a web browser:

$ godoc -http=:8080  

The following screenshot shows the kind of output you get on a web browser after visiting http://localhost:8080/pkg/ while the previous command is running. You can use any port number you want, provided that it is not already in use:

The most important tool for a programmer is the editor they use for writing the source code. When I am on a Mac, I typically use the TextMate editor, but when I am on a different Unix machine, I prefer vi. Choosing an editor is not an easy task because you are going to spend a lot of time with it. However, any text editor will do the job as long as it does not put any control characters inside the source code files. The following screenshot shows the TextMate editor in action:

Go is not perfect but it has some very interesting features. The list of the Go strong features includes the following:

• Go code is easy to read and easy to understand.
• Go wants happy developers because a happy developer writes better code!
• The Go compiler prints practical warning and error messages that help you solve the actual problem. Putting it simply, the Go compiler is there to help you, not to make your life difficult!
• Go code is portable.
• Go is a modern programming language.
• Go has support for procedural, concurrent, and distributed programming.
• Go supports Garbage Collection (GC) so you do not have to deal with memory allocation and deallocation. However, GC might slow down your programs a little.
• Go does not have a preprocessor and does high-speed compilation. Consequently, Go can be used as a scripting language.
• Go can build web applications. Building a web application in C is simply not very efficient unless you use a nonstandard external library. Additionally, Go provides programmers with a simple web server for testing purposes.
• The standard Go library offers many packages that simplify the work of the programmer. Additionally, the methods found in the standard Go library are tested and debugged in advance, which means that most of the time they contain no bugs.
• Go uses static linking by default, which means that the produced binary files can be easily transferred to other machines with the same OS. Consequently, the developer does not need to worry about libraries, dependencies, and different library versions.
• You will not need a GUI for developing, debugging, and testing Go applications as Go can be used from the command line.
• Go supports Unicode. This means that you do not need any extra code to print characters from multiple human languages.
• Go keeps concepts orthogonal because a few orthogonal features work better than many overlapping ones.

The list of Go disadvantages includes the following:

• Well, Go is not C, which means that you or your team should learn a new programming language to develop systems software.
• Go does not have direct support for object-oriented programming, which can be a problem for programmers that are used to writing code in an object-oriented manner. Nevertheless, you can use composition in Go to mimic inheritance.
• Back when Unix was first introduced, C was the only programming language for writing systems software. Nowadays, you can also use Rust, C++, and Swift for writing systems software, which means that not everybody will be using Go.
• C is still faster than any other programming language for systems programming mainly because Unix is written in C.

Strictly speaking, a process is an execution environment that contains instructions, user-data and system-data parts, and other kinds of resources that are obtained during runtime. A program is a file that contains instructions and data, which are used for initializing the instruction and user-data parts of a process.

Back when the Unix operating system was first introduced, computers had single CPUs without multiple cores and a small amount of RAM. However, Unix was a multiuser and multitasking operating system. In order to actually be a multiuser and do multitasking, it had to be able to run each individual process sporadically, which means that a process should have multiple states. The following figure shows the possible states of a process as well as the right path to go from one state to another:

There are three categories of processes: user processes, Kernel processes, and Daemon processes:

• User processes run in user space and usually have no special access rights
• Kernel processes are being executed in kernel space only and can fully access all kernel data structures
• Daemon processes are programs that can be found in the user space and run in the background without the need for a Terminal

Realizing that you cannot control the state of a process is really important, as this is the job of the scheduler of the operating system that runs in the kernel. Putting it simply, you cannot tell when the state of a process is going to change or when the process is going to go into the running state, so your code cannot count on any such assumptions!

1. Visit the Go website: https://golang.org/.
2. Install Go on your system and find out its version.
3. Type the code of the Hello World program on your own and save it to a file.
4. If you are on a Mac, download TextMate from http://macromates.com/.
5. If you are on a Mac, download the TextWrangler editor from http://www.barebones.com/products/TextWrangler/ and try it.
6. Try to learn vi or Emacs on your own if you are not already familiar with another Unix text editor.
7. Look at any Go code you can find and try to make small changes to it.

In this chapter, you learned how to get Go on your computer, the features of the latest Go version, the advantages and disadvantages of Go, and the gofmt and godoc Go tools, as well as some important things about the Unix operating system.

The next chapter will not only tell you how to compile your Go code but it will also discuss other important Go topics such as reading and using command-line arguments, environment variables, writing functions, data structures, interfaces, getting user input, and printing output.