Go provides excellent support for both basic and complex I/O. The recipes in this chapter will explore common Go interfaces that are used to deal with I/O and show you how to make use of them. The Go standard library frequently uses these interfaces, and they will be used by recipes throughout the book.

You'll learn how to work with data in memory and in the form of streams. You'll see examples of working with files, directories, and the CSV format. The temporary files recipe looks at a mechanism to work with files without the overhead of dealing with name collision and more. Lastly, we'll explore Go standard templates for both plain text and HTML.

These recipes should lay the foundation for the use of interfaces to represent and modify data, and should help you think about data in an abstract and flexible way.

In this chapter, the following recipes will be covered:

• Using the common I/O interfaces
• Using the bytes and strings packages
• Working with directories and files
• Working with the CSV format
• Working with temporary files
• Working with text/template and html/template

In order to proceed with all the recipes in this chapter, configure your environment according to these steps:

1. Download and install Go 1.12.6 or greater on your operating system at https://golang.org/doc/install.
2. Open a Terminal or console application and create and navigate to a project directory, such as ~/projects/go-programming-cookbook. All code will be run and modified from this directory.
3. Clone the latest code into~/projects/go-programming-cookbook-original, as shown in the following code. It is recommended that you work from that directory rather than typing the examples manually:

          $ git clone git@github.com:PacktPublishing/Go-Programming-Cookbook-Second-Edition.git go-programming-cookbook-original
        

The Go language provides a number of I/O interfaces that are used throughout the standard library. It is best practice to make use of these interfaces wherever possible rather than passing structures or other types directly. Two powerful interfaces we will explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library, and understanding how to use them will make you a better Go developer.

The Reader and Writer interfaces look like this:

type Reader interface {
        Read(p []byte) (n int, err error)
}

type Writer interface {
        Write(p []byte) (n int, err error)
}

Go also makes it easy to combine interfaces. For example, take a look at the following code:

type Seeker interface {
        Seek(offset int64, whence int) (int64, error)
}

type ReadSeeker interface {
        Reader
        Seeker
}

This recipe will also explore an io function called Pipe(), as shown in the following code:

func Pipe() (*PipeReader, *PipeWriter)

The remainder of this book will make use of these interfaces.

How to do it...

The following steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/interfaces.
2. Navigate to this directory.
3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/interfaces

        

You should see a file called go.mod that contains the following:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/interfaces    

4. Copy tests from ~/projects/go-programming-cookbook-original/chapter1/interfaces or use this as an exercise to write some of your own code!

5. Create a file called interfaces.go with the following contents:

        package interfaces

        import (
                "fmt"
                "io"
                "os"
        )

        // Copy copies data from in to out first directly,
        // then using a buffer. It also writes to stdout
        func Copy(in io.ReadSeeker, out io.Writer) error {
                // we write to out, but also Stdout
                w := io.MultiWriter(out, os.Stdout)

                // a standard copy, this can be dangerous if there's a 
                // lot of data in in
                if _, err := io.Copy(w, in); err != nil {
                    return err
                }

                in.Seek(0, 0)

                // buffered write using 64 byte chunks
                buf := make([]byte, 64)
                if _, err := io.CopyBuffer(w, in, buf); err != nil {
                    return err
                }

                // lets print a new line
                fmt.Println()

                return nil
        }

6. Create a file called pipes.go with the following contents:

        package interfaces

        import (
                "io"
                "os"
        )

        // PipeExample helps give some more examples of using io  
        //interfaces
        func PipeExample() error {
                // the pipe reader and pipe writer implement
                // io.Reader and io.Writer
                r, w := io.Pipe()

                // this needs to be run in a separate go routine
                // as it will block waiting for the reader
                // close at the end for cleanup
                go func() {
                    // for now we'll write something basic,
                    // this could also be used to encode json
                    // base64 encode, etc.
                    w.Write([]byte("test\n"))
                    w.Close()
                }()

                if _, err := io.Copy(os.Stdout, r); err != nil {
                    return err
                }
                return nil
        }

7. Create a new directory named example and navigate to it.
8. Create a main.gofile with the following contents:

        package main

        import (
             "bytes"
             "fmt"

             "github.com/PacktPublishing/
              Go-Programming-Cookbook-Second-Edition/
              chapter1/bytestrings"
        )

        func main() {
                in := bytes.NewReader([]byte("example"))
                out := &bytes.Buffer{}
                fmt.Print("stdout on Copy = ")
                if err := interfaces.Copy(in, out); err != nil {
                        panic(err)
                }

                fmt.Println("out bytes buffer =", out.String())

                fmt.Print("stdout on PipeExample = ")
                if err := interfaces.PipeExample(); err != nil {
                        panic(err)
                }
        }

9. Run go run ..

10. You may also run the following:

          $ go build 

          $ ./example
        

You should see the following output:

          $ go run .
          

          stdout on Copy = exampleexample
          

          out bytes buffer = exampleexample
          

          stdout on PipeExample = test
        

11. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

The Copy() function copies bytes between interfaces and treats that data like a stream. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a MultiWriter interface that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface was used instead, rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. You can also use a buffered write if your stream is not fitted into the memory.

The PipeReader and PipeWriter structures implement the io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.

Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused, but provides great flexibility with thread safety when linking input and output streams.

The bytes and strings packages have a number of useful helpers to work with and convert the data from string to byte types, and vice versa. They allow the creation of buffers that work with a number of common I/O interfaces.

How to do it...

The following steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/bytestrings.
2. Navigate to this directory.
3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/bytestrings

        

You should see a file called go.mod that contains the following content:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/bytestrings    

4. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/bytestrings or use this as an exercise to write some of your own code!
5. Create a file calledbuffer.gowith the following contents:

        package bytestrings

        import (
                "bytes"
                "io"
                "io/ioutil"
        )

        // Buffer demonstrates some tricks for initializing bytes    
        //Buffers
        // These buffers implement an io.Reader interface
        func Buffer(rawString string) *bytes.Buffer {

                // we'll start with a string encoded into raw bytes
                rawBytes := []byte(rawString)

                // there are a number of ways to create a buffer from 
                // the raw bytes or from the original string
                var b = new(bytes.Buffer)
                b.Write(rawBytes)

                // alternatively
                b = bytes.NewBuffer(rawBytes)

                // and avoiding the initial byte array altogether
                b = bytes.NewBufferString(rawString)

                return b
        }

        // ToString is an example of taking an io.Reader and consuming 
        // it all, then returning a string
        func toString(r io.Reader) (string, error) {
                b, err := ioutil.ReadAll(r)
                if err != nil {
                    return "", err
                }
                return string(b), nil
        }

6. Create a file calledbytes.gowith the following contents:

        package bytestrings

        import (
                "bufio"
                "bytes"
                "fmt"
        )

        // WorkWithBuffer will make use of the buffer created by the
        // Buffer function
        func WorkWithBuffer() error {
                rawString := "it's easy to encode unicode into a byte 
                              array"

                b := Buffer(rawString)

                // we can quickly convert a buffer back into byes with
                // b.Bytes() or a string with b.String()
                fmt.Println(b.String())

                // because this is an io Reader we can make use of  
                // generic io reader functions such as
                s, err := toString(b)
                if err != nil {
                    return err
                }
                fmt.Println(s)

                // we can also take our bytes and create a bytes reader
                // these readers implement io.Reader, io.ReaderAt, 
                // io.WriterTo, io.Seeker, io.ByteScanner, and 
                // io.RuneScanner interfaces
                reader := bytes.NewReader([]byte(rawString))

                // we can also plug it into a scanner that allows 
                // buffered reading and tokenization
                scanner := bufio.NewScanner(reader)
                scanner.Split(bufio.ScanWords)

                // iterate over all of the scan events
                for scanner.Scan() {
                    fmt.Print(scanner.Text())
                }

                return nil
        }

7. Create a file called string.go with the following contents:

        package bytestrings

        import (
                "fmt"
                "io"
                "os"
                "strings"
        )

        // SearchString shows a number of methods
        // for searching a string
        func SearchString() {
                s := "this is a test"

                // returns true because s contains
                // the word this
                fmt.Println(strings.Contains(s, "this"))

                // returns true because s contains the letter a
                // would also match if it contained b or c
                fmt.Println(strings.ContainsAny(s, "abc"))

                // returns true because s starts with this
                fmt.Println(strings.HasPrefix(s, "this"))

                // returns true because s ends with this
                fmt.Println(strings.HasSuffix(s, "test"))
                }

        // ModifyString modifies a string in a number of ways
        func ModifyString() {
                s := "simple string"

                // prints [simple string]
                fmt.Println(strings.Split(s, " "))

                // prints "Simple String"
                fmt.Println(strings.Title(s))

                // prints "simple string"; all trailing and
                // leading white space is removed
                s = " simple string "
                fmt.Println(strings.TrimSpace(s))
        }

        // StringReader demonstrates how to create
        // an io.Reader interface quickly with a string
        func StringReader() {
                s := "simple string\n"
                r := strings.NewReader(s)

                // prints s on Stdout
                io.Copy(os.Stdout, r)
        }

8. Create a new directory named example and navigate to it.
9. Create a main.go file with the following contents:

        package main

        import "github.com/PacktPublishing/
                Go-Programming-Cookbook-Second-Edition/
                chapter1/bytestrings"

        func main() {
                err := bytestrings.WorkWithBuffer()
                if err != nil {
                        panic(err)
                }

                // each of these print to stdout
                bytestrings.SearchString()
                bytestrings.ModifyString()
                bytestrings.StringReader() 
        }

10. Run go run ..
11. You may also run the following:

          $ go build

          $ ./example
        

You should see the following output:

          $ go run .
          

          it's easy to encode unicode into a byte array ??
          

          it's easy to encode unicode into a byte array ??
          

          it'seasytoencodeunicodeintoabytearray??true
          

          true
          

          true
          

          true
          

          [simple string]
          

          Simple String
          

          simple string
          

          simple string
        

12. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

The bytes library provides a number of convenience functions when working with data. A buffer, for example, is far more flexible than an array of bytes when working with stream-processing libraries or methods. Once you've created a buffer, it can be used to satisfy an io.Reader interface so that you can take advantage of ioutil functions to manipulate the data. For streaming applications, you'd probably want to use a buffer and a scanner. The bufio package comes in handy for these cases. Sometimes, using an array or slice is more appropriate for smaller datasets or when you have a lot of memory on your machine.

Go provides a lot of flexibility in converting data between interfaces when using these basic types—it's relatively simple to convert between strings and bytes. When working with strings, thestringspackage provides a number of convenience functions to work with, search, and manipulate strings. In some cases, a good regular expression may be appropriate, but most of the time, thestringsandstrconvpackages are sufficient. Thestringspackage allows you to make a string look like a title, split it into an array, or trim whitespace. It also provides aReaderinterface of its own that can be used instead of thebytespackage reader type.

Working with directories and files can be difficult when you switch between platforms (Windows and Linux, for example). Go provides cross-platform support to work with files and directories in the os and ioutils packages. We've already seen examples of ioutils, but now we'll explore how to use them in another way!

How to do it...

The following steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/filedirs.
2. Navigate to this directory.

3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/filedirs

        

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/filedirs

4. Copy tests from~/projects/go-programming-cookbook-original/chapter1/filedirs or use this as an exercise to write some of your own code!
5. Create a file called dirs.go with the following contents:

        package filedirs

        import (
                "errors"
                "io"
                "os"
        )

        // Operate manipulates files and directories
        func Operate() error {
                // this 0755 is similar to what you'd see with Chown
                // on a command line this will create a director 
                // /tmp/example, you may also use an absolute path 
                // instead of a relative one
                if err := os.Mkdir("example_dir", os.FileMode(0755)); 
                err !=  nil {
                        return err
                }

                // go to the /tmp directory
                if err := os.Chdir("example_dir"); err != nil {
                        return err
                }

                // f is a generic file object
                // it also implements multiple interfaces
                // and can be used as a reader or writer
                // if the correct bits are set when opening
                f, err := os.Create("test.txt")
                if err != nil {
                        return err
                }

                // we write a known-length value to the file and 
                // validate that it wrote correctly
                value := []byte("hellon")
                count, err := f.Write(value)
                if err != nil {
                        return err
                }
                if count != len(value) {
                        return errors.New("incorrect length returned 
                        from write")
                }

                if err := f.Close(); err != nil {
                        return err
                }

                // read the file
                f, err = os.Open("test.txt")
                if err != nil {
                        return err
                }

                io.Copy(os.Stdout, f)

                if err := f.Close(); err != nil {
                        return err
                }

                // go to the /tmp directory
                if err := os.Chdir(".."); err != nil {
                        return err
                }

                // cleanup, os.RemoveAll can be dangerous if you
                // point at the wrong directory, use user input,
                // and especially if you run as root
                if err := os.RemoveAll("example_dir"); err != nil {
                        return err
                }

                return nil
        }

6. Create a file called files.gowith the following contents:

        package filedirs

        import (
                "bytes"
                "io"
                "os"
                "strings"
        )

        // Capitalizer opens a file, reads the contents,
        // then writes those contents to a second file
                func Capitalizer(f1 *os.File, f2 *os.File) error {
                if _, err := f1.Seek(0, io.SeekStart); err != nil {
                        return err
                }

                var tmp = new(bytes.Buffer)

                if _, err := io.Copy(tmp, f1); err != nil {
                        return err
                }

                s := strings.ToUpper(tmp.String())

                if _, err := io.Copy(f2, strings.NewReader(s)); err != 
                nil {
                        return err
                }
                return nil
        }

        // CapitalizerExample creates two files, writes to one
        //then calls Capitalizer() on both
        func CapitalizerExample() error {
                f1, err := os.Create("file1.txt")
                if err != nil {
                        return err
                }

                if _, err := f1.Write([]byte(`this file contains a 
                number of words and new lines`)); err != nil {
                        return err
                }

                f2, err := os.Create("file2.txt")
                if err != nil {
                        return err
                }

                if err := Capitalizer(f1, f2); err != nil {
                        return err
                }

                if err := os.Remove("file1.txt"); err != nil {
                        return err
                }

                if err := os.Remove("file2.txt"); err != nil {
                        return err
                }

                return nil
        }

7. Create a new directory named example and navigate to it.

8. Create a main.go file with the following contents:

        package main

        import "github.com/PacktPublishing/
                Go-Programming-Cookbook-Second-Edition/
                chapter1/filedirs"

        func main() {
                if err := filedirs.Operate(); err != nil {
                        panic(err)
                }

                if err := filedirs.CapitalizerExample(); err != nil {
                        panic(err)
                }
        }

9. Run go run ..
10. You may also run the following:

          $ go build

          $ ./example
        

You should see the following output:

          $ go run . 
          

          hello
        

11. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

If you're familiar with files in Unix, the Go os library should feel very familiar. You can perform basically all common operations—Stat a file to collect attributes, collect a file with different permissions, and create and modify directories and files. In this recipe, we performed a number of manipulations to directories and files and then cleaned up after ourselves.

Working with file objects is very similar to working with in-memory streams. Files also provide a number of convenience functions directly, such as Chown, Stat, and Truncate. The easiest way to get comfortable with files is to make use of them. In all the previous recipes, we have to be careful to clean up after our programs.

Working with files is a very common operation when building backend applications. Files can be used for configuration, secret keys, as temporary storage, and more. Go wraps OS system calls using the os package and allows the same functions to operate regardless of whether you're using Windows or Unix.

Once your file is opened and stored in a File structure, it can easily be passed into a number of interfaces (we discussed these interfaces earlier). All the earlier examples can use os.File structures directly instead of buffers and in-memory data streams in order to operate on data stored on the disk . This may be useful for certain techniques, such as writing all logs to stderr and the file at the same time with a single write call.

CSV is a common format that is used to manipulate data. It's common, for example, to import or export a CSV file into Excel. The Go CSV package operates on data interfaces, and as a result, it's easy to write data to a buffer, stdout, a file, or a socket. The examples in this section will show some common ways to get data into and out of the CSV format.

How to do it...

These steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/csvformat.
2. Navigate to this directory.
3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/csvformat

        

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/csvformat

4. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/csvformat or use this as an exercise to write some of your own code!
5. Create a file called read_csv.gowith the following contents:

        package csvformat

        import (
                "bytes"
                "encoding/csv"
                "fmt"
                "io"
                "strconv"
        )

        // Movie will hold our parsed CSV
        type Movie struct {
                Title string
                Director string
                Year int
        }

        // ReadCSV gives shows some examples of processing CSV
        // that is passed in as an io.Reader
        func ReadCSV(b io.Reader) ([]Movie, error) {

                r := csv.NewReader(b)

                // These are some optional configuration options
                r.Comma = ';'
                r.Comment = '-'

                var movies []Movie

                // grab and ignore the header for now
                // we may also want to use this for a dictionary key or
                // some other form of lookup
                _, err := r.Read()
                if err != nil && err != io.EOF {
                        return nil, err
                }

                // loop until it's all processed
                for {
                        record, err := r.Read()
                        if err == io.EOF {
                                break
                        } else if err != nil {
                                return nil, err
                        }

                        year, err := strconv.ParseInt(record[2], 10, 
                        64)
                        if err != nil {
                                return nil, err
                        }

                        m := Movie{record[0], record[1], int(year)}
                        movies = append(movies, m)
                }
                return movies, nil
        }

6. Add this additional function to read_csv.go, as follows:

        // AddMoviesFromText uses the CSV parser with a string
        func AddMoviesFromText() error {
                // this is an example of us taking a string, converting
                // it into a buffer, and reading it 
                // with the csv package
                in := `
                - first our headers
                movie title;director;year released

                - then some data
                Guardians of the Galaxy Vol. 2;James Gunn;2017
                Star Wars: Episode VIII;Rian Johnson;2017
                `

                b := bytes.NewBufferString(in)
                m, err := ReadCSV(b)
                if err != nil {
                        return err
                }
                fmt.Printf("%#vn", m)
                return nil
        }

7. Create a file called write_csv.gowith thefollowing contents:

        package csvformat

        import (
                "bytes"
                "encoding/csv"
                "io"
                "os"
        )

        // A Book has an Author and Title
        type Book struct {
                Author string
                Title string
        }

        // Books is a named type for an array of books
        type Books []Book

        // ToCSV takes a set of Books and writes to an io.Writer
        // it returns any errors
        func (books *Books) ToCSV(w io.Writer) error {
                n := csv.NewWriter(w)
                err := n.Write([]string{"Author", "Title"})
                if err != nil {
                        return err
                }
                for _, book := range *books {
                        err := n.Write([]string{book.Author, 
                        book.Title})
                        if err != nil {
                                return err
                        }
                }

                n.Flush()
                return n.Error()
        }

8. Add these additional functions to write_csv.go, as follows:

        // WriteCSVOutput initializes a set of books
        // and writes them to os.Stdout
        func WriteCSVOutput() error {
                b := Books{
                        Book{
                                Author: "F Scott Fitzgerald",
                                Title: "The Great Gatsby",
                        },
                        Book{
                                Author: "J D Salinger",
                                Title: "The Catcher in the Rye",
                        },
                }

                return b.ToCSV(os.Stdout)
        }

        // WriteCSVBuffer returns a buffer CSV for
        // a set of books
        func WriteCSVBuffer() (*bytes.Buffer, error) {
                b := Books{
                        Book{
                                Author: "F Scott Fitzgerald",
                                Title: "The Great Gatsby",
                        },
                        Book{
                                Author: "J D Salinger",
                                Title: "The Catcher in the Rye",
                        },
                }

                w := &bytes.Buffer{}
                err := b.ToCSV(w)
                return w, err
        }

9. Create a new directory named example and navigate to it.
10. Create a main.go file with the following contents:

        package main

        import (
                "fmt"

                "github.com/PacktPublishing/
                 Go-Programming-Cookbook-Second-Edition/
                 chapter1/csvformat"
        )

        func main() {
                if err := csvformat.AddMoviesFromText(); err != nil {
                        panic(err)
                }

                if err := csvformat.WriteCSVOutput(); err != nil {
                        panic(err)
                }

                buffer, err := csvformat.WriteCSVBuffer()
                if err != nil {
                        panic(err)
                }

                fmt.Println("Buffer = ", buffer.String())
        }

11. Rungo run ..
12. You may also run the following:

          $ go build
          

          $ ./example
        

You should see the following output:

          $ go run . 
          

          []csvformat.Movie{csvformat.Movie{Title:"Guardians of the 
Galaxy Vol. 2", Director:"James Gunn", Year:2017},         
csvformat.Movie{Title:"Star Wars: Episode VIII", Director:"Rian 
Johnson", Year:2017}}
          

          Author,Title
          

          F Scott Fitzgerald,The Great Gatsby
          

          J D Salinger,The Catcher in the Rye
          

          Buffer = Author,Title
          

          F Scott Fitzgerald,The Great Gatsby
          

          J D Salinger,The Catcher in the Rye
        

13. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

In order to learn how to read a CSV format, we first represent our data as a structure. In Go, it's very useful to format data as a structure, as it makes things such as marshaling and encoding relatively simple. Our read example uses movies as our data type. The function takes anio.Readerinterface that holds our CSV data as an input. This could be a file or a buffer. We then use that data to create and populate aMoviestructure, including converting the year into an integer. We also add options to the CSV parser to use; (semi-colon) as the separator and- (hyphen) as a comment line.

Next, we explore the same idea, but in reverse. Novels are represented with a title and an author. We initialize an array of novels and then write specific novels in the CSV format to an io.Writer interface. Once again, this can be a file, stdout, or a buffer.

The CSV package is an excellent example of why you'd want to think of data flows in Go as implementing common interfaces. It's easy to change the source and destination of our data with small one-line tweaks, and we can easily manipulate CSV data without using an excessive amount of memory or time. For example, it would be possible to read from a stream of data one record at a time and write to a separate stream in a modified format one record at a time. Doing this would not incur significant memory or processor usage.

Later, when we explore data pipelines and worker pools, you'll see how these ideas can be combined and how to handle these streams in parallel.

We've created and made use of files for a number of examples so far. We've also had to manually deal with cleanup, name collision, and more. Temporary files and directories are a quicker, simpler way to handle these cases.

How to do it...

The following steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/tempfiles.
2. Navigate to this directory.
3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/tempfiles

        

You should see a file called go.mod that contains the following contents:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/tempfiles    

4. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/tempfiles or use this as an exercise to write some of your own code!
5. Create a file calledtemp_files.gowith the following contents:

        package tempfiles

        import (
                "fmt"
                "io/ioutil"
                "os"
        )

        // WorkWithTemp will give some basic patterns for working
        // with temporary files and directories
        func WorkWithTemp() error {
                // If you need a temporary place to store files with 
                // the same name ie. template1-10.html a temp directory 
                //  is a good way to approach it, the first argument 
                // being blank means it will use create the directory                
                // in the location returned by 
                // os.TempDir()
                t, err := ioutil.TempDir("", "tmp")
                if err != nil {
                        return err
                }

                // This will delete everything inside the temp file 
                // when this function exits if you want to do this 
                //  later, be sure to return the directory name to the 
                // calling function
                defer os.RemoveAll(t)

                // the directory must exist to create the tempfile
                // created. t is an *os.File object.
                tf, err := ioutil.TempFile(t, "tmp")
                if err != nil {
                        return err
                }

                fmt.Println(tf.Name())

                // normally we'd delete the temporary file here, but 
                // because we're placing it in a temp directory, it 
                // gets cleaned up by the earlier defer

                return nil
        }

6. Create a new directory named example and navigate to it.
7. Create a main.go file with the following contents:

        package main

        import "github.com/PacktPublishing/
                Go-Programming-Cookbook-Second-Edition/
                chapter1/tempfiles"

        func main() {
                if err := tempfiles.WorkWithTemp(); err != nil {
                        panic(err)
                }
        }

8. Run go run ..
9. You may also run the following:

          $ go build

          $ ./example
        

You should see the following output (with a different path):

          $ go run . 
          

          /var/folders/kd/ygq5l_0d1xq1lzk_c7htft900000gn/T
/tmp764135258/tmp588787953
        

10. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

Creating temporary files and directories can be done using the ioutil package. Although you must still delete the files yourself, using RemoveAll is the convention, and it will do this for you with only one extra line of code.

When writing tests, the use of temporary files is highly recommended. It's also useful for things such as build artifacts and more. The Go ioutil package will try and honor the OS preferences by default, but it allows you to fall back to other directories if required.

Go provides rich support for templates. It is simple to nest templates, import functions, represent variables, iterate over data, and so on. If you need something more sophisticated than a CSV writer, templates may be a great solution.

Another application for templates is for websites. When we want to render server-side data to the client, templates fit the bill nicely. At first, Go templates can appear confusing. This section will explore working with templates, collecting templates inside of a directory, and working with HTML templates.

How to do it...

These steps cover how to write and run your application:

1. From your Terminal or console application, create a new directory called ~/projects/go-programming-cookbook/chapter1/templates.
2. Navigate to this directory.

3. Run the following command:

          $ go mod init github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/templates

        

You should see a file called go.mod that contains the following content:

module github.com/PacktPublishing/Go-Programming-Cookbook-Second-Edition/chapter1/templates

4. Copy the tests from~/projects/go-programming-cookbook-original/chapter1/templates or use this as an exercise to write some of your own code!
5. Create a file called templates.go with the following contents:

        package templates

        import (
                "os"
                "strings"
                "text/template"
        )

        const sampleTemplate = `
                This template demonstrates printing a {{ .Variable | 
                printf "%#v" }}.

                {{if .Condition}}
                If condition is set, we'll print this
                {{else}}
                Otherwise, we'll print this instead
                {{end}}

                Next we'll iterate over an array of strings:
                {{range $index, $item := .Items}}
                {{$index}}: {{$item}}
                {{end}}

                We can also easily import other functions like 
                strings.Split
                then immediately used the array created as a result:
                {{ range $index, $item := split .Words ","}}
                {{$index}}: {{$item}}
                {{end}}

                Blocks are a way to embed templates into one another
                {{ block "block_example" .}}
                No Block defined!
                {{end}}


                {{/*
                This is a way
                to insert a multi-line comment
                */}}
`

        const secondTemplate = `
                {{ define "block_example" }}
                {{.OtherVariable}}
                {{end}}
`

6. Add a function to the end of templates.go, as follows:

        // RunTemplate initializes a template and demonstrates a 
        // variety of template helper functions
        func RunTemplate() error {
                data := struct {
                        Condition bool
                        Variable string
                        Items []string
                        Words string
                        OtherVariable string
                }{
                        Condition: true,
                        Variable: "variable",
                        Items: []string{"item1", "item2", "item3"},
                        Words: 
                        "another_item1,another_item2,another_item3",
                        OtherVariable: "I'm defined in a second 
                        template!",
                }

                funcmap := template.FuncMap{
                        "split": strings.Split,
                }

                // these can also be chained
                t := template.New("example")
                t = t.Funcs(funcmap)

                // We could use Must instead to panic on error
                // template.Must(t.Parse(sampleTemplate))
                t, err := t.Parse(sampleTemplate)
                if err != nil {
                        return err
                }

                // to demonstrate blocks we'll create another template
                // by cloning the first template, then parsing a second
                t2, err := t.Clone()
                if err != nil {
                        return err
                }

                t2, err = t2.Parse(secondTemplate)
                if err != nil {
                        return err
                }

                // write the template to stdout and populate it
                // with data
                err = t2.Execute(os.Stdout, &data)
                if err != nil {
                        return err
                }

                return nil
        }

7. Create a file called template_files.go with the following contents:

        package templates

        import (
                "io/ioutil"
                "os"
                "path/filepath"
                "text/template"
        )

        //CreateTemplate will create a template file that contains data
        func CreateTemplate(path string, data string) error {
                return ioutil.WriteFile(path, []byte(data), 
                os.FileMode(0755))
        }

        // InitTemplates sets up templates from a directory
        func InitTemplates() error {
                tempdir, err := ioutil.TempDir("", "temp")
                if err != nil {
                        return err
                }
                defer os.RemoveAll(tempdir)

                err = CreateTemplate(filepath.Join(tempdir, "t1.tmpl"), 
                `Template 1! {{ .Var1 }}
                {{ block "template2" .}} {{end}}
                {{ block "template3" .}} {{end}}
                `)
                if err != nil {
                        return err
                }

                err = CreateTemplate(filepath.Join(tempdir, "t2.tmpl"), 
                `{{ define "template2"}}Template 2! {{ .Var2 }}{{end}}
                `)
                if err != nil {
                        return err
                }

                err = CreateTemplate(filepath.Join(tempdir, "t3.tmpl"), 
                `{{ define "template3"}}Template 3! {{ .Var3 }}{{end}}
                `)
                if err != nil {
                        return err
                }

                pattern := filepath.Join(tempdir, "*.tmpl")

                // Parse glob will combine all the files that match 
                // glob and combine them into a single template
                tmpl, err := template.ParseGlob(pattern)
                if err != nil {
                        return err
                }

                // Execute can also work with a map instead
                // of a struct
                tmpl.Execute(os.Stdout, map[string]string{
                        "Var1": "Var1!!",
                        "Var2": "Var2!!",
                        "Var3": "Var3!!",
                 })

                 return nil
        }

8. Create a file called html_templates.go with the following contents:

        package templates

        import (
                "fmt"
                "html/template"
                "os"
        )

        // HTMLDifferences highlights some of the differences
        // between html/template and text/template
        func HTMLDifferences() error {
                t := template.New("html")
                t, err := t.Parse("<h1>Hello! {{.Name}}</h1>n")
                if err != nil {
                        return err
         }

                // html/template auto-escapes unsafe operations like 
                // javascript injection this is contextually aware and 
                // will behave differently
                // depending on where a variable is rendered
                err = t.Execute(os.Stdout, map[string]string{"Name": "                 
                      <script>alert('Can you see me?')</script>"})
                if err != nil {
                        return err
                }

                // you can also manually call the escapers
                fmt.Println(template.JSEscaper(`example         
                <example@example.com>`))
                fmt.Println(template.HTMLEscaper(`example 
                <example@example.com>`))
                fmt.Println(template.URLQueryEscaper(`example 
                <example@example.com>`))

                return nil
        }

9. Create a new directory named example and navigate to it.
10. Create a main.go file with the following contents:

        package main

        import "github.com/PacktPublishing/
                Go-Programming-Cookbook-Second-Edition/
                chapter1/templates"

        func main() {
                if err := templates.RunTemplate(); err != nil {
                        panic(err)
                }

                if err := templates.InitTemplates(); err != nil {
                        panic(err)
                }

                if err := templates.HTMLDifferences(); err != nil {
                        panic(err)
                }
        }

11. Run go run ..
12. You may also run the following:

          $ go build

          $ ./example
        

You should see the following output (with a different path):

13. If you copied or wrote your own tests, go up one directory and run go test, and ensure that all tests pass.

How it works...

Go has two template packages: text/template and html/template. They share functionality and a variety of functions. In general, you should use html/template to render websites and text/template for everything else. Templates are plain text, but variables and functions can be used inside of curly brace blocks.

The template packages also provide convenience methods to work with files. The example that we used here creates a number of templates in a temporary directory and then reads them all with a single line of code.

The html/template package is a wrapper around the text/template package. All of the template examples work with the html/template package directly, using no modification and only changing the import statement. HTML templates provide the added benefit of context-aware safety; this prevents security breaches such as JavaScript injection.

The template packages provide what you'd expect from a modern template library. It's easy to combine templates, add application logic, and ensure safety when emitting results to HTML and JavaScript.