Go Programming Cookbook - Second Edition

1 (1 reviews total)
By Aaron Torres
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. I/O and Filesystems

About this book

Go (or Golang) is a statically typed programming language developed at Google. Known for its vast standard library, it also provides features such as garbage collection, type safety, dynamic-typing capabilities, and additional built-in types. This book will serve as a reference while implementing Go features to build your own applications.

This Go cookbook helps you put into practice the advanced concepts and libraries that Golang offers. The recipes in the book follow best practices such as documentation, testing, and vendoring with Go modules, as well as performing clean abstractions using interfaces. You'll learn how code works and the common pitfalls to watch out for. The book covers basic type and error handling, and then moves on to explore applications, such as websites, command-line tools, and filesystems, that interact with users. You'll even get to grips with parallelism, distributed systems, and performance tuning.

By the end of the book, you'll be able to use open source code and concepts in Go programming to build enterprise-class applications without any hassle.

Publication date:
July 2019
Publisher
Packt
Pages
434
ISBN
9781789800982

 
I/O and Filesystems

Go provides excellent support for both basic and complex I/O. The recipes in this chapter will explore common Go interfaces that are used to deal with I/O and show you how to make use of them. The Go standard library frequently uses these interfaces, and they will be used by recipes throughout the book.

You'll learn how to work with data in memory and in the form of streams. You'll see examples of working with files, directories, and the CSV format. The temporary files recipe looks at a mechanism to work with files without the overhead of dealing with name collision and more. Lastly, we'll explore Go standard templates for both plain text and HTML.

These recipes should lay the foundation for the use of interfaces to represent and modify data, and should help you think about data in an abstract and flexible way.

In this chapter, the following recipes will be covered:

  • Using the common I/O interfaces
  • Using the bytes and strings packages
  • Working with directories and files
  • Working with the CSV format
  • Working with temporary files
  • Working with text/template and html/template
 

Technical requirements

In order to proceed with all the recipes in this chapter, configure your environment according to these steps:

  1. Download and install Go 1.12.6 or greater on your operating system at https://golang.org/doc/install.
  2. Open a Terminal or console application and create and navigate to a project directory, such as ~/projects/go-programming-cookbook. All code will be run and modified from this directory.
  3. Clone the latest code into~/projects/go-programming-cookbook-original, as shown in the following code. It is recommended that you work from that directory rather than typing the examples manually:
          $ git clone [email protected]:PacktPublishing/Go-Programming-Cookbook-Second-Edition.git go-programming-cookbook-original
        
 

Using the common I/O interfaces

The Go language provides a number of I/O interfaces that are used throughout the standard library. It is best practice to make use of these interfaces wherever possible rather than passing structures or other types directly. Two powerful interfaces we will explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library, and understanding how to use them will make you a better Go developer.

The Reader and Writer interfaces look like this:

type Reader interface {
Read(p []byte) (n int, err error)
}

type Writer interface {
Write(p []byte) (n int, err error)
}

Go also makes it easy to combine interfaces. For example, take a look at the following code:

type Seeker interface {
Seek(offset int64, whence int) (int64, error)
}

type ReadSeeker interface {
Reader
Seeker
}

This recipe will also explore an io function called Pipe(), as shown in the following code:

func Pipe() (*PipeReader, *PipeWriter)

The remainder of this book will make use of these interfaces.

 

How to do it...

The following steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/interfaces.
  2. Navigate to this directory.
  3. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/interfaces

You should see a file called go.mod that contains the following:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/interfaces    
  1. Copy tests from ~/projects/go-programming-cookbook-original/chapter1/interfaces or use this as an exercise to write some of your own code!
  1. Create a file called interfaces.go with the following contents:
        package interfaces

import (
"fmt"
"io"
"os"
)

// Copy copies data from in to out first directly,
// then using a buffer. It also writes to stdout
func Copy(in io.ReadSeeker, out io.Writer) error {
// we write to out, but also Stdout
w := io.MultiWriter(out, os.Stdout)

// a standard copy, this can be dangerous if there's a
// lot of data in in
if _, err := io.Copy(w, in); err != nil {
return err
}

in.Seek(0, 0)

// buffered write using 64 byte chunks
buf := make([]byte, 64)
if _, err := io.CopyBuffer(w, in, buf); err != nil {
return err
}

// lets print a new line
fmt.Println()

return nil
}
  1. Create a file called pipes.go with the following contents:
        package interfaces

import (
"io"
"os"
)

// PipeExample helps give some more examples of using io
//interfaces
func PipeExample() error {
// the pipe reader and pipe writer implement
// io.Reader and io.Writer
r, w := io.Pipe()

// this needs to be run in a separate go routine
// as it will block waiting for the reader
// close at the end for cleanup
go func() {
// for now we'll write something basic,
// this could also be used to encode json
// base64 encode, etc.
w.Write([]byte("test\n"))
w.Close()
}()

if _, err := io.Copy(os.Stdout, r); err != nil {
return err
}
return nil
}
  1. Create a new directory named example and navigate to it.
  2. Create a main.gofile with the following contents:
        package main

import (
"bytes"
"fmt"

"github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/bytestrings"
)

func main() {
in := bytes.NewReader([]byte("example"))
out := &bytes.Buffer{}
fmt.Print("stdout on Copy = ")
if err := interfaces.Copy(in, out); err != nil {
panic(err)
}

fmt.Println("out bytes buffer =", out.String())

fmt.Print("stdout on PipeExample = ")
if err := interfaces.PipeExample(); err != nil {
panic(err)
}
}
  1. Run go run ..
  1. You may also run the following:
          $ go build 
$ ./example

You should see the following output:

          $ go run .
          
stdout on Copy = exampleexample
out bytes buffer = exampleexample
stdout on PipeExample = test
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

The Copy() function copies bytes between interfaces and treats that data like a stream. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a MultiWriter interface that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface was used instead, rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. You can also use a buffered write if your stream is not fitted into the memory.

The PipeReader and PipeWriter structures implement the io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.

Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused, but provides great flexibility with thread safety when linking input and output streams.

 

Using the bytes and strings packages

The bytes and strings packages have a number of useful helpers to work with and convert the data from string to byte types, and vice versa. They allow the creation of buffers that work with a number of common I/O interfaces.

 

How to do it...

The following steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/bytestrings.
  2. Navigate to this directory.
  3. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/bytestrings

You should see a file called go.mod that contains the following content:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/bytestrings    
  1. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/bytestrings or use this as an exercise to write some of your own code!
  2. Create a file calledbuffer.gowith the following contents:
        package bytestrings

import (
"bytes"
"io"
"io/ioutil"
)

// Buffer demonstrates some tricks for initializing bytes
//Buffers
// These buffers implement an io.Reader interface
func Buffer(rawString string) *bytes.Buffer {

// we'll start with a string encoded into raw bytes
rawBytes := []byte(rawString)

// there are a number of ways to create a buffer from
// the raw bytes or from the original string
var b = new(bytes.Buffer)
b.Write(rawBytes)

// alternatively
b = bytes.NewBuffer(rawBytes)

// and avoiding the initial byte array altogether
b = bytes.NewBufferString(rawString)

return b
}

// ToString is an example of taking an io.Reader and consuming
// it all, then returning a string
func toString(r io.Reader) (string, error) {
b, err := ioutil.ReadAll(r)
if err != nil {
return "", err
}
return string(b), nil
}
  1. Create a file calledbytes.gowith the following contents:
        package bytestrings

import (
"bufio"
"bytes"
"fmt"
)

// WorkWithBuffer will make use of the buffer created by the
// Buffer function
func WorkWithBuffer() error {
rawString := "it's easy to encode unicode into a byte
array"

b := Buffer(rawString)

// we can quickly convert a buffer back into byes with
// b.Bytes() or a string with b.String()
fmt.Println(b.String())

// because this is an io Reader we can make use of
// generic io reader functions such as
s, err := toString(b)
if err != nil {
return err
}
fmt.Println(s)

// we can also take our bytes and create a bytes reader
// these readers implement io.Reader, io.ReaderAt,
// io.WriterTo, io.Seeker, io.ByteScanner, and
// io.RuneScanner interfaces
reader := bytes.NewReader([]byte(rawString))

// we can also plug it into a scanner that allows
// buffered reading and tokenization
scanner := bufio.NewScanner(reader)
scanner.Split(bufio.ScanWords)

// iterate over all of the scan events
for scanner.Scan() {
fmt.Print(scanner.Text())
}

return nil
}
  1. Create a file called string.go with the following contents:
        package bytestrings

import (
"fmt"
"io"
"os"
"strings"
)

// SearchString shows a number of methods
// for searching a string
func SearchString() {
s := "this is a test"

// returns true because s contains
// the word this
fmt.Println(strings.Contains(s, "this"))

// returns true because s contains the letter a
// would also match if it contained b or c
fmt.Println(strings.ContainsAny(s, "abc"))

// returns true because s starts with this
fmt.Println(strings.HasPrefix(s, "this"))

// returns true because s ends with this
fmt.Println(strings.HasSuffix(s, "test"))
}

// ModifyString modifies a string in a number of ways
func ModifyString() {
s := "simple string"

// prints [simple string]
fmt.Println(strings.Split(s, " "))

// prints "Simple String"
fmt.Println(strings.Title(s))

// prints "simple string"; all trailing and
// leading white space is removed
s = " simple string "
fmt.Println(strings.TrimSpace(s))
}

// StringReader demonstrates how to create
// an io.Reader interface quickly with a string
func StringReader() {
s := "simple string\n"
r := strings.NewReader(s)

// prints s on Stdout
io.Copy(os.Stdout, r)
}
  1. Create a new directory named example and navigate to it.
  2. Create a main.go file with the following contents:
        package main

import "github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/bytestrings"

func main() {
err := bytestrings.WorkWithBuffer()
if err != nil {
panic(err)
}

// each of these print to stdout
bytestrings.SearchString()
bytestrings.ModifyString()
bytestrings.StringReader()
}
  1. Run go run ..
  2. You may also run the following:
          $ go build
$ ./example

You should see the following output:

          $ go run .
          
it's easy to encode unicode into a byte array ??
it's easy to encode unicode into a byte array ??
it'seasytoencodeunicodeintoabytearray??true
true
true
true
[simple string]
Simple String
simple string
simple string
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

The bytes library provides a number of convenience functions when working with data. A buffer, for example, is far more flexible than an array of bytes when working with stream-processing libraries or methods. Once you've created a buffer, it can be used to satisfy an io.Reader interface so that you can take advantage of ioutil functions to manipulate the data. For streaming applications, you'd probably want to use a buffer and a scanner. The bufio package comes in handy for these cases. Sometimes, using an array or slice is more appropriate for smaller datasets or when you have a lot of memory on your machine.

Go provides a lot of flexibility in converting data between interfaces when using these basic types—it's relatively simple to convert between strings and bytes. When working with strings, thestringspackage provides a number of convenience functions to work with, search, and manipulate strings. In some cases, a good regular expression may be appropriate, but most of the time, thestringsandstrconvpackages are sufficient. Thestringspackage allows you to make a string look like a title, split it into an array, or trim whitespace. It also provides aReaderinterface of its own that can be used instead of thebytespackage reader type.

 

Working with directories and files

Working with directories and files can be difficult when you switch between platforms (Windows and Linux, for example). Go provides cross-platform support to work with files and directories in the os and ioutils packages. We've already seen examples of ioutils, but now we'll explore how to use them in another way!

 

How to do it...

The following steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/filedirs.
  2. Navigate to this directory.
  1. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/filedirs

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/filedirs
  1. Copy tests from~/projects/go-programming-cookbook-original/chapter1/filedirs or use this as an exercise to write some of your own code!
  2. Create a file called dirs.go with the following contents:
        package filedirs

import (
"errors"
"io"
"os"
)

// Operate manipulates files and directories
func Operate() error {
// this 0755 is similar to what you'd see with Chown
// on a command line this will create a director
// /tmp/example, you may also use an absolute path
// instead of a relative one
if err := os.Mkdir("example_dir", os.FileMode(0755));
err != nil {
return err
}

// go to the /tmp directory
if err := os.Chdir("example_dir"); err != nil {
return err
}

// f is a generic file object
// it also implements multiple interfaces
// and can be used as a reader or writer
// if the correct bits are set when opening
f, err := os.Create("test.txt")
if err != nil {
return err
}

// we write a known-length value to the file and
// validate that it wrote correctly
value := []byte("hellon")
count, err := f.Write(value)
if err != nil {
return err
}
if count != len(value) {
return errors.New("incorrect length returned
from write")
}

if err := f.Close(); err != nil {
return err
}

// read the file
f, err = os.Open("test.txt")
if err != nil {
return err
}

io.Copy(os.Stdout, f)

if err := f.Close(); err != nil {
return err
}

// go to the /tmp directory
if err := os.Chdir(".."); err != nil {
return err
}

// cleanup, os.RemoveAll can be dangerous if you
// point at the wrong directory, use user input,
// and especially if you run as root
if err := os.RemoveAll("example_dir"); err != nil {
return err
}

return nil
}
  1. Create a file called files.gowith the following contents:
        package filedirs

import (
"bytes"
"io"
"os"
"strings"
)

// Capitalizer opens a file, reads the contents,
// then writes those contents to a second file
func Capitalizer(f1 *os.File, f2 *os.File) error {
if _, err := f1.Seek(0, io.SeekStart); err != nil {
return err
}

var tmp = new(bytes.Buffer)

if _, err := io.Copy(tmp, f1); err != nil {
return err
}

s := strings.ToUpper(tmp.String())

if _, err := io.Copy(f2, strings.NewReader(s)); err !=
nil {
return err
}
return nil
}

// CapitalizerExample creates two files, writes to one
//then calls Capitalizer() on both
func CapitalizerExample() error {
f1, err := os.Create("file1.txt")
if err != nil {
return err
}

if _, err := f1.Write([]byte(`this file contains a
number of words and new lines`)); err != nil {
return err
}

f2, err := os.Create("file2.txt")
if err != nil {
return err
}

if err := Capitalizer(f1, f2); err != nil {
return err
}

if err := os.Remove("file1.txt"); err != nil {
return err
}

if err := os.Remove("file2.txt"); err != nil {
return err
}

return nil
}
  1. Create a new directory named example and navigate to it.
  1. Create a main.go file with the following contents:
        package main

import "github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/filedirs"

func main() {
if err := filedirs.Operate(); err != nil {
panic(err)
}

if err := filedirs.CapitalizerExample(); err != nil {
panic(err)
}
}
  1. Run go run ..
  2. You may also run the following:
          $ go build
$ ./example

You should see the following output:

          $ go run . 
          
hello
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

If you're familiar with files in Unix, the Go os library should feel very familiar. You can perform basically all common operations—Stat a file to collect attributes, collect a file with different permissions, and create and modify directories and files. In this recipe, we performed a number of manipulations to directories and files and then cleaned up after ourselves.

Working with file objects is very similar to working with in-memory streams. Files also provide a number of convenience functions directly, such as Chown, Stat, and Truncate. The easiest way to get comfortable with files is to make use of them. In all the previous recipes, we have to be careful to clean up after our programs.

Working with files is a very common operation when building backend applications. Files can be used for configuration, secret keys, as temporary storage, and more. Go wraps OS system calls using the os package and allows the same functions to operate regardless of whether you're using Windows or Unix.

Once your file is opened and stored in a File structure, it can easily be passed into a number of interfaces (we discussed these interfaces earlier). All the earlier examples can use os.File structures directly instead of buffers and in-memory data streams in order to operate on data stored on the disk . This may be useful for certain techniques, such as writing all logs to stderr and the file at the same time with a single write call.

 

Working with the CSV format

CSV is a common format that is used to manipulate data. It's common, for example, to import or export a CSV file into Excel. The Go CSV package operates on data interfaces, and as a result, it's easy to write data to a buffer, stdout, a file, or a socket. The examples in this section will show some common ways to get data into and out of the CSV format.

 

How to do it...

These steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/csvformat.
  2. Navigate to this directory.
  3. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/csvformat

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/csvformat
  1. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/csvformat or use this as an exercise to write some of your own code!
  2. Create a file called read_csv.gowith the following contents:
        package csvformat

import (
"bytes"
"encoding/csv"
"fmt"
"io"
"strconv"
)

// Movie will hold our parsed CSV
type Movie struct {
Title string
Director string
Year int
}

// ReadCSV gives shows some examples of processing CSV
// that is passed in as an io.Reader
func ReadCSV(b io.Reader) ([]Movie, error) {

r := csv.NewReader(b)

// These are some optional configuration options
r.Comma = ';'
r.Comment = '-'

var movies []Movie

// grab and ignore the header for now
// we may also want to use this for a dictionary key or
// some other form of lookup
_, err := r.Read()
if err != nil && err != io.EOF {
return nil, err
}

// loop until it's all processed
for {
record, err := r.Read()
if err == io.EOF {
break
} else if err != nil {
return nil, err
}

year, err := strconv.ParseInt(record[2], 10,
64)
if err != nil {
return nil, err
}

m := Movie{record[0], record[1], int(year)}
movies = append(movies, m)
}
return movies, nil
}
  1. Add this additional function to read_csv.go, as follows:
        // AddMoviesFromText uses the CSV parser with a string
func AddMoviesFromText() error {
// this is an example of us taking a string, converting
// it into a buffer, and reading it
// with the csv package
in := `
- first our headers
movie title;director;year released

- then some data
Guardians of the Galaxy Vol. 2;James Gunn;2017
Star Wars: Episode VIII;Rian Johnson;2017
`

b := bytes.NewBufferString(in)
m, err := ReadCSV(b)
if err != nil {
return err
}
fmt.Printf("%#vn", m)
return nil
}
  1. Create a file called write_csv.gowith thefollowing contents:
        package csvformat

import (
"bytes"
"encoding/csv"
"io"
"os"
)

// A Book has an Author and Title
type Book struct {
Author string
Title string
}

// Books is a named type for an array of books
type Books []Book

// ToCSV takes a set of Books and writes to an io.Writer
// it returns any errors
func (books *Books) ToCSV(w io.Writer) error {
n := csv.NewWriter(w)
err := n.Write([]string{"Author", "Title"})
if err != nil {
return err
}
for _, book := range *books {
err := n.Write([]string{book.Author,
book.Title})
if err != nil {
return err
}
}

n.Flush()
return n.Error()
}
  1. Add these additional functions to write_csv.go, as follows:
        // WriteCSVOutput initializes a set of books
// and writes them to os.Stdout
func WriteCSVOutput() error {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}

return b.ToCSV(os.Stdout)
}

// WriteCSVBuffer returns a buffer CSV for
// a set of books
func WriteCSVBuffer() (*bytes.Buffer, error) {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}

w := &bytes.Buffer{}
err := b.ToCSV(w)
return w, err
}
  1. Create a new directory named example and navigate to it.
  2. Create a main.go file with the following contents:
        package main

import (
"fmt"

"github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/csvformat"
)

func main() {
if err := csvformat.AddMoviesFromText(); err != nil {
panic(err)
}

if err := csvformat.WriteCSVOutput(); err != nil {
panic(err)
}

buffer, err := csvformat.WriteCSVBuffer()
if err != nil {
panic(err)
}

fmt.Println("Buffer = ", buffer.String())
}
  1. Rungo run ..
  2. You may also run the following:
          $ go build
          
$ ./example

You should see the following output:

          $ go run . 
          
[]csvformat.Movie{csvformat.Movie{Title:"Guardians of the
Galaxy Vol. 2", Director:"James Gunn", Year:2017},
csvformat.Movie{Title:"Star Wars: Episode VIII", Director:"Rian
Johnson", Year:2017}}

Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
Buffer = Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

In order to learn how to read a CSV format, we first represent our data as a structure. In Go, it's very useful to format data as a structure, as it makes things such as marshaling and encoding relatively simple. Our read example uses movies as our data type. The function takes anio.Readerinterface that holds our CSV data as an input. This could be a file or a buffer. We then use that data to create and populate aMoviestructure, including converting the year into an integer. We also add options to the CSV parser to use; (semi-colon) as the separator and- (hyphen) as a comment line.

Next, we explore the same idea, but in reverse. Novels are represented with a title and an author. We initialize an array of novels and then write specific novels in the CSV format to an io.Writer interface. Once again, this can be a file, stdout, or a buffer.

The CSV package is an excellent example of why you'd want to think of data flows in Go as implementing common interfaces. It's easy to change the source and destination of our data with small one-line tweaks, and we can easily manipulate CSV data without using an excessive amount of memory or time. For example, it would be possible to read from a stream of data one record at a time and write to a separate stream in a modified format one record at a time. Doing this would not incur significant memory or processor usage.

Later, when we explore data pipelines and worker pools, you'll see how these ideas can be combined and how to handle these streams in parallel.

 

Working with temporary files

We've created and made use of files for a number of examples so far. We've also had to manually deal with cleanup, name collision, and more. Temporary files and directories are a quicker, simpler way to handle these cases.

 

How to do it...

The following steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/tempfiles.
  2. Navigate to this directory.
  3. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/tempfiles

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/tempfiles    
  1. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/tempfiles or use this as an exercise to write some of your own code!
  2. Create a file calledtemp_files.gowith the following contents:
        package tempfiles

import (
"fmt"
"io/ioutil"
"os"
)

// WorkWithTemp will give some basic patterns for working
// with temporary files and directories
func WorkWithTemp() error {
// If you need a temporary place to store files with
// the same name ie. template1-10.html a temp directory
// is a good way to approach it, the first argument
// being blank means it will use create the directory
// in the location returned by
// os.TempDir()
t, err := ioutil.TempDir("", "tmp")
if err != nil {
return err
}

// This will delete everything inside the temp file
// when this function exits if you want to do this
// later, be sure to return the directory name to the
// calling function
defer os.RemoveAll(t)

// the directory must exist to create the tempfile
// created. t is an *os.File object.
tf, err := ioutil.TempFile(t, "tmp")
if err != nil {
return err
}

fmt.Println(tf.Name())

// normally we'd delete the temporary file here, but
// because we're placing it in a temp directory, it
// gets cleaned up by the earlier defer

return nil
}
  1. Create a new directory named example and navigate to it.
  2. Create a main.go file with the following contents:
        package main

import "github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/tempfiles"

func main() {
if err := tempfiles.WorkWithTemp(); err != nil {
panic(err)
}
}
  1. Run go run ..
  2. You may also run the following:
          $ go build
$ ./example

You should see the following output (with a different path):

          $ go run . 
          
/var/folders/kd/ygq5l_0d1xq1lzk_c7htft900000gn/T
/tmp764135258/tmp588787953
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

Creating temporary files and directories can be done using the ioutil package. Although you must still delete the files yourself, using RemoveAll is the convention, and it will do this for you with only one extra line of code.

When writing tests, the use of temporary files is highly recommended. It's also useful for things such as build artifacts and more. The Go ioutil package will try and honor the OS preferences by default, but it allows you to fall back to other directories if required.

 

Working with text/template and html/template

Go provides rich support for templates. It is simple to nest templates, import functions, represent variables, iterate over data, and so on. If you need something more sophisticated than a CSV writer, templates may be a great solution.

Another application for templates is for websites. When we want to render server-side data to the client, templates fit the bill nicely. At first, Go templates can appear confusing. This section will explore working with templates, collecting templates inside of a directory, and working with HTML templates.

 

How to do it...

These steps cover how to write and run your application:

  1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/templates.
  2. Navigate to this directory.
  1. Run the following command:
          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/templates

You should see a file called go.mod that contains the following content:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/templates
  1. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/templates or use this as an exercise to write some of your own code!
  2. Create a file called templates.go with the following contents:
        package templates

import (
"os"
"strings"
"text/template"
)

const sampleTemplate = `
This template demonstrates printing a {{ .Variable |
printf "%#v" }}.

{{if .Condition}}
If condition is set, we'll print this
{{else}}
Otherwise, we'll print this instead
{{end}}

Next we'll iterate over an array of strings:
{{range $index, $item := .Items}}
{{$index}}: {{$item}}
{{end}}

We can also easily import other functions like
strings.Split
then immediately used the array created as a result:
{{ range $index, $item := split .Words ","}}
{{$index}}: {{$item}}
{{end}}

Blocks are a way to embed templates into one another
{{ block "block_example" .}}
No Block defined!
{{end}}


{{/*
This is a way
to insert a multi-line comment
*/}}
`

const secondTemplate = `
{{ define "block_example" }}
{{.OtherVariable}}
{{end}}
`
  1. Add a function to the end of templates.go, as follows:
        // RunTemplate initializes a template and demonstrates a 
// variety of template helper functions
func RunTemplate() error {
data := struct {
Condition bool
Variable string
Items []string
Words string
OtherVariable string
}{
Condition: true,
Variable: "variable",
Items: []string{"item1", "item2", "item3"},
Words:
"another_item1,another_item2,another_item3",
OtherVariable: "I'm defined in a second
template!",
}

funcmap := template.FuncMap{
"split": strings.Split,
}

// these can also be chained
t := template.New("example")
t = t.Funcs(funcmap)

// We could use Must instead to panic on error
// template.Must(t.Parse(sampleTemplate))
t, err := t.Parse(sampleTemplate)
if err != nil {
return err
}

// to demonstrate blocks we'll create another template
// by cloning the first template, then parsing a second
t2, err := t.Clone()
if err != nil {
return err
}

t2, err = t2.Parse(secondTemplate)
if err != nil {
return err
}

// write the template to stdout and populate it
// with data
err = t2.Execute(os.Stdout, &data)
if err != nil {
return err
}

return nil
}
  1. Create a file called template_files.go with the following contents:
        package templates

import (
"io/ioutil"
"os"
"path/filepath"
"text/template"
)

//CreateTemplate will create a template file that contains data
func CreateTemplate(path string, data string) error {
return ioutil.WriteFile(path, []byte(data),
os.FileMode(0755))
}

// InitTemplates sets up templates from a directory
func InitTemplates() error {
tempdir, err := ioutil.TempDir("", "temp")
if err != nil {
return err
}
defer os.RemoveAll(tempdir)

err = CreateTemplate(filepath.Join(tempdir, "t1.tmpl"),
`Template 1! {{ .Var1 }}
{{ block "template2" .}} {{end}}
{{ block "template3" .}} {{end}}
`)
if err != nil {
return err
}

err = CreateTemplate(filepath.Join(tempdir, "t2.tmpl"),
`{{ define "template2"}}Template 2! {{ .Var2 }}{{end}}
`)
if err != nil {
return err
}

err = CreateTemplate(filepath.Join(tempdir, "t3.tmpl"),
`{{ define "template3"}}Template 3! {{ .Var3 }}{{end}}
`)
if err != nil {
return err
}

pattern := filepath.Join(tempdir, "*.tmpl")

// Parse glob will combine all the files that match
// glob and combine them into a single template
tmpl, err := template.ParseGlob(pattern)
if err != nil {
return err
}

// Execute can also work with a map instead
// of a struct
tmpl.Execute(os.Stdout, map[string]string{
"Var1": "Var1!!",
"Var2": "Var2!!",
"Var3": "Var3!!",
})

return nil
}
  1. Create a file called html_templates.go with the following contents:
        package templates

import (
"fmt"
"html/template"
"os"
)

// HTMLDifferences highlights some of the differences
// between html/template and text/template
func HTMLDifferences() error {
t := template.New("html")
t, err := t.Parse("<h1>Hello! {{.Name}}</h1>n")
if err != nil {
return err
}

// html/template auto-escapes unsafe operations like
// javascript injection this is contextually aware and
// will behave differently
// depending on where a variable is rendered
err = t.Execute(os.Stdout, map[string]string{"Name": "
<script>alert('Can you see me?')</script>"})
if err != nil {
return err
}

// you can also manually call the escapers
fmt.Println(template.JSEscaper(`example
<[email protected]>`))
fmt.Println(template.HTMLEscaper(`example
<[email protected]>`))
fmt.Println(template.URLQueryEscaper(`example
<[email protected]>`))

return nil
}
  1. Create a new directory named example and navigate to it.
  2. Create a main.go file with the following contents:
        package main

import "github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter1/templates"

func main() {
if err := templates.RunTemplate(); err != nil {
panic(err)
}

if err := templates.InitTemplates(); err != nil {
panic(err)
}

if err := templates.HTMLDifferences(); err != nil {
panic(err)
}
}
  1. Run go run ..
  2. You may also run the following:
          $ go build
$ ./example

You should see the following output (with a different path):

  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.
 

How it works...

Go has two template packages: text/template and html/template. They share functionality and a variety of functions. In general, you should use html/template to render websites and text/template for everything else. Templates are plain text, but variables and functions can be used inside of curly brace blocks.

The template packages also provide convenience methods to work with files. The example that we used here creates a number of templates in a temporary directory and then reads them all with a single line of code.

The html/template package is a wrapper around the text/template package. All of the template examples work with the html/template package directly, using no modification and only changing the import statement. HTML templates provide the added benefit of context-aware safety; this prevents security breaches such as JavaScript injection.

The template packages provide what you'd expect from a modern template library. It's easy to combine templates, add application logic, and ensure safety when emitting results to HTML and JavaScript.

About the Author

  • Aaron Torres

    Aaron Torres received his master's degree in computer science from the New Mexico Institute of Mining and Technology. He has worked on distributed systems in high-performance computing and in large-scale web and microservices applications. He currently leads a team of Go developers that refines and focuses on Go best practices with an emphasis on continuous delivery and automated testing. Aaron has published a number of papers and has several patents in the area of storage and I/O. He is passionate about sharing his knowledge and ideas with others. He is also a huge fan of the Go language and open source for backend systems and development.

    Browse publications by this author

Latest Reviews

(1 reviews total)
I didn't receive my order by email. I'm stay confuse if my ebook was send or no.

Recommended For You

Mastering Go - Second Edition

Dive deep into the Go language and become an expert Go developer

By Mihalis Tsoukalos
Hands-On System Programming with Go

Explore the fundamentals of systems programming starting from kernel API and filesystem to network programming and process communications

By Alex Guerrieri
The Go Workshop

Cut through the noise and get real results with a step-by-step approach to learning Go programming

By Delio D'Anna and 5 more
Hands-On Microservices with Kubernetes

Enhance your skills in building scalable infrastructure for your cloud-based applications

By Gigi Sayfan