Download code from GitHub
I/O and File Systems
In this chapter, the following recipes will be covered:
- Using the common I/O interfaces
- Using the bytes and strings packages
- Working with directories and files
- Working with the CSV format
- Working with temporary files
- Working with text/template and HTML/templates
Introduction
Go provides excellent support for both basic and complex I/O. The recipes in this chapter will explore common Go interfaces to deal with I/O and show how to make use of them. The Go standard library frequently uses these interfaces, and these interfaces will be used by recipes throughout the book.
You'll learn how to work with data in memory and in the form of streams. You'll see examples of working with files and directories and of working with the CSV format. The temporary files recipe discusses a mechanism to work with files without the overhead of dealing with name collision and more. Lastly, we'll explore Go standard templates for both plain text and HTML.
These recipes should lay the foundation for the use of interfaces to represent and modify data and should help you think about data in an abstract and flexible way.
Using the common I/O interfaces
Go provides a number of I/O interfaces used throughout the standard library. It is a best practice to make use of these interfaces wherever possible rather than passing structs or other types directly. Two powerful interfaces we explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library and understanding how to use them will make you a better Go developer.
The Reader and Writer interfaces look like this:
type Reader interface {
Read(p []byte) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
Go also makes it easy to combine interfaces. For example, take a look at the following code:
type Seeker interface {
Seek(offset int64, whence int) (int64, error)
}
type ReadSeeker interface {
Reader
Seeker
}
The recipe will also explore an io function called Pipe():
func Pipe() (*PipeReader, *PipeWriter)
The remainder of this book will make use of these interfaces.
Getting ready
Configure your environment according to these steps:
- Download and install Go on your operating system at https://golang.org/doc/install and configure your GOPATH environment variable.
- Open a terminal/console application, navigate to your GOPATH/src directory, and create a project directory such as $GOPATH/src/github.com/yourusername/customrepo.
All code will be run and modified from this directory.
- Optionally, install the latest tested version of the code using the following command:
go get github.com/agtorre/go-cookbook/
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/interfaces.
- Navigate to that directory.
Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/interfaces, or use this as an exercise to write some of your own code.
- Create a file called interfaces.go with the following contents:
package interfaces
import (
"fmt"
"io"
"os"
)
// Copy copies data from in to out first directly,
// then using a buffer. It also writes to stdout
func Copy(in io.ReadSeeker, out io.Writer) error {
// we write to out, but also Stdout
w := io.MultiWriter(out, os.Stdout)
// a standard copy, this can be dangerous if there's a
// lot of data in in
if _, err := io.Copy(w, in); err != nil {
return err
}
in.Seek(0, 0)
// buffered write using 64 byte chunks
buf := make([]byte, 64)
if _, err := io.CopyBuffer(w, in, buf); err != nil {
return err
}
// lets print a new line
fmt.Println()
return nil
}
- Create a file called pipes.go with the following contents:
package interfaces
import (
"io"
"os"
)
// PipeExample helps give some more examples of using io
//interfaces
func PipeExample() error {
// the pipe reader and pipe writer implement
// io.Reader and io.Writer
r, w := io.Pipe()
// this needs to be run in a separate go routine
// as it will block waiting for the reader
// close at the end for cleanup
go func() {
// for now we'll write something basic,
// this could also be used to encode json
// base64 encode, etc.
w.Write([]byte("testn"))
w.Close()
}()
if _, err := io.Copy(os.Stdout, r); err != nil {
return err
}
return nil
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:
package main
import (
"bytes"
"fmt"
"github.com/agtorre/go-cookbook/chapter1/interfaces"
)
func main() {
in := bytes.NewReader([]byte("example"))
out := &bytes.Buffer{}
fmt.Print("stdout on Copy = ")
if err := interfaces.Copy(in, out); err != nil {
panic(err)
}
fmt.Println("out bytes buffer =", out.String())
fmt.Print("stdout on PipeExample = ")
if err := interfaces.PipeExample(); err != nil {
panic(err)
}
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see the following output:
$ go run main.go
stdout on Copy = exampleexample
out bytes buffer = exampleexample
stdout on PipeExample = test
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
The Copy() function copies between interfaces and treats them like streams. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a multi-writer that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface were used instead rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. There's also an example of a buffered write that you might use if your stream is not fit into the memory.
The PipeReader and PipeWriter structs implement io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.
Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused but provides great flexibility with thread-safety when linking input and output streams.
Using the bytes and strings packages
The bytes and string packages have a number of useful helpers to work with and convert between strings and byte types. They allow the creation of buffers that work with a number of common I/O interfaces.
Getting ready
Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/bytestrings.
- Navigate to this directory.
- Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/bytesstrings, or use this as an exercise to write some of your own code!
- Create a file called buffer.go with the following contents:
package bytestrings
import (
"bytes"
"io"
"io/ioutil"
)
// Buffer demonstrates some tricks for initializing bytes
//Buffers
// These buffers implement an io.Reader interface
func Buffer(rawString string) *bytes.Buffer {
// we'll start with a string encoded into raw bytes
rawBytes := []byte(rawString)
// there are a number of ways to create a buffer from
// the raw bytes or from the original string
var b = new(bytes.Buffer)
b.Write(rawBytes)
// alternatively
b = bytes.NewBuffer(rawBytes)
// and avoiding the intial byte array altogether
b = bytes.NewBufferString(rawString)
return b
}
// ToString is an example of taking an io.Reader and consuming
// it all, then returning a string
func toString(r io.Reader) (string, error) {
b, err := ioutil.ReadAll(r)
if err != nil {
return "", err
}
return string(b), nil
}
- Create a file called bytes.go with the following contents:
package bytestrings
import (
"bufio"
"bytes"
"fmt"
)
// WorkWithBuffer will make use of the buffer created by the
// Buffer function
func WorkWithBuffer() error {
rawString := "it's easy to encode unicode into a byte
array"
b := Buffer(rawString)
// we can quickly convert a buffer back into byes with
// b.Bytes() or a string with b.String()
fmt.Println(b.String())
// because this is an io Reader we can make use of
// generic io reader functions such as
s, err := toString(b)
if err != nil {
return err
}
fmt.Println(s)
// we can also take our bytes and create a bytes reader
// these readers implement io.Reader, io.ReaderAt,
// io.WriterTo, io.Seeker, io.ByteScanner, and
// io.RuneScanner interfaces
reader := bytes.NewReader([]byte(rawString))
// we can also plug it into a scanner that allows
// buffered reading and tokenzation
scanner := bufio.NewScanner(reader)
scanner.Split(bufio.ScanWords)
// iterate over all of the scan events
for scanner.Scan() {
fmt.Print(scanner.Text())
}
return nil
}
- Create a file called string.go with the following contents:
package bytestrings
import (
"fmt"
"io"
"os"
"strings"
)
// SearchString shows a number of methods
// for searching a string
func SearchString() {
s := "this is a test"
// returns true because s contains
// the word this
fmt.Println(strings.Contains(s, "this"))
// returns true because s contains the letter a
// would also match if it contained b or c
fmt.Println(strings.ContainsAny(s, "abc"))
// returns true because s starts with this
fmt.Println(strings.HasPrefix(s, "this"))
// returns true because s ends with this
fmt.Println(strings.HasSuffix(s, "test"))
}
// ModifyString modifies a string in a number of ways
func ModifyString() {
s := "simple string"
// prints [simple string]
fmt.Println(strings.Split(s, " "))
// prints "Simple String"
fmt.Println(strings.Title(s))
// prints "simple string"; all trailing and
// leading white space is removed
s = " simple string "
fmt.Println(strings.TrimSpace(s))
}
// StringReader demonstrates how to create
// an io.Reader interface quickly with a string
func StringReader() {
s := "simple stringn"
r := strings.NewReader(s)
// prints s on Stdout
io.Copy(os.Stdout, r)
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:
package main
import "github.com/agtorre/go-cookbook/chapter1/bytestrings"
func main() {
err := bytestrings.WorkWithBuffer()
if err != nil {
panic(err)
}
// each of these print to stdout
bytestrings.SearchString()
bytestrings.ModifyString()
bytestrings.StringReader()
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see the following output:
$ go run main.go
it's easy to encode unicode into a byte array ??
it's easy to encode unicode into a byte array ??
it'seasytoencodeunicodeintoabytearray??true
true
true
true
[simple string]
Simple String
simple string
simple string
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
The bytes library provides a number of convenience functions when working with data. A buffer, for example, is far more flexible than an array of bytes when working with stream processing libraries or methods. Once you've created a buffer, it can be used to satisfy an io.Reader interface so you can take advantage of ioutil functions to manipulate the data. For steaming applications, you'd probably want to use a buffer and a scanner. The bufio package comes in handy for these cases. Sometimes, using an array or slice is more appropriate for smaller datasets or when you have a lot of memory on your machine.
Go provides a lot of flexibility in converting between interfaces with these basic types--it's relatively simple to convert between strings and bytes. When working with strings, the strings package provides a number of convenience functions to work with, search, and manipulate strings. In some cases, a good regular expression may be appropriate, but most of the time, the strings and strconv packages are sufficient. The strings package allows you to make a string look like a title, split it into an array, or trim whitespace. It also provides a Reader interface of its own that can be used instead of the bytes package reader type.
Working with directories and files
Working with directories and files can be difficult when you switch between platforms (Windows and Linux, for example). Go provides cross-platform support to work with files and directories in the os and ioutils packages. We've already seen examples of ioutils, but now we'll explore how to use them in another way!
Getting ready
Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/filedirs.
- Navigate to this directory.
- Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/filedirs, or use this as an exercise to write some of your own code!
- Create a file called dirs.go with the following contents:
package filedirs
import (
"errors"
"io"
"os"
)
// Operate manipulates files and directories
func Operate() error {
// this 0777 is similar to what you'd see with chown
// on a command line this will create a director
// /tmp/example, you may also use an absolute path
// instead of a relative one
if err := os.Mkdir("example_dir", os.FileMode(0755));
err != nil {
return err
}
// go to the /tmp directory
if err := os.Chdir("example_dir"); err != nil {
return err
}
// f is a generic file object
// it also implements multiple interfaces
// and can be used as a reader or writer
// if the correct bits are set when opening
f, err := os.Create("test.txt")
if err != nil {
return err
}
// we write a known-length value to the file and
// validate that it wrote correctly
value := []byte("hellon")
count, err := f.Write(value)
if err != nil {
return err
}
if count != len(value) {
return errors.New("incorrect length returned
from write")
}
if err := f.Close(); err != nil {
return err
}
// read the file
f, err = os.Open("test.txt")
if err != nil {
return err
}
io.Copy(os.Stdout, f)
if err := f.Close(); err != nil {
return err
}
// go to the /tmp directory
if err := os.Chdir(".."); err != nil {
return err
}
// cleanup, os.RemoveAll can be dangerous if you
// point at the wrong directory, use user input,
// and especially if you run as root
if err := os.RemoveAll("example_dir"); err != nil {
return err
}
return nil
}
- Create a file called bytes.go with the following contents:
package filedirs
import (
"bytes"
"io"
"os"
"strings"
)
// Capitalizer opens a file, reads the contents,
// then writes those contents to a second file
func Capitalizer(f1 *os.File, f2 *os.File) error {
if _, err := f1.Seek(0, 0); err != nil {
return err
}
var tmp = new(bytes.Buffer)
if _, err := io.Copy(tmp, f1); err != nil {
return err
}
s := strings.ToUpper(tmp.String())
if _, err := io.Copy(f2, strings.NewReader(s)); err !=
nil {
return err
}
return nil
}
// CapitalizerExample creates two files, writes to one
//then calls Capitalizer() on both
func CapitalizerExample() error {
f1, err := os.Create("file1.txt")
if err != nil {
return err
}
if _, err := f1.Write([]byte(`this file contains a
number of words and new lines`)); err != nil {
return err
}
f2, err := os.Create("file2.txt")
if err != nil {
return err
}
if err := Capitalizer(f1, f2); err != nil {
return err
}
if err := os.Remove("file1.txt"); err != nil {
return err
}
if err := os.Remove("file2.txt"); err != nil {
return err
}
return nil
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the filedirs package import to use the path you set up in step 2:
package main
import "github.com/agtorre/go-cookbook/chapter1/filedirs"
func main() {
if err := filedirs.Operate(); err != nil {
panic(err)
}
if err := filedirs.CapitalizerExample(); err != nil {
panic(err)
}
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see the following output:
$ go run main.go
hello
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
If you're familiar with files in Unix, the Go os library should feel very familiar. You can do basically all common operations--stat a file to collect attributes, collect a file with different permissions, and create and modify directories and files. We performed a number of manipulations to directories and files and then cleaned up after ourselves.
Working with file objects is very similar to in-memory streams. Files also provide a number of convenience functions directly, such as Chown, Stat, and Truncate. The easiest way to get comfortable with files is to make use of them. In all the previous recipes, we have to be careful to clean up after our programs.
Working with files is a very common operation when building backend applications. Files can be used for configuration, secret keys, as temporary storage, and more. Go wraps OS system calls using the os package and allows the same functions to operate regardless of whether you're using Windows or Unix.
Once your file is opened and stored in a File struct, it can easily be passed into a number of interfaces discussed earlier. All the earlier examples of working with buffers and in-memory data streams can be replaced directly with file objects. This may be useful for things such as writing all logs to stderr and the file at the same time with a single write call.
Working with the CSV format
CSV is a common format to manipulate data. It's common, for example, to import or export a CSV file into Excel. The Go CSV package operates on data interfaces, and as a result, it's easy to write data to a buffer, stdout, a file, or to a socket. The examples in this section will show some common ways to get data into and out of the CSV format.
Getting ready
Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/csvformat.
- Navigate to this directory.
- Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/csvformat, or use this as an exercise to write some of your own code!
- Create a file called read_csv.go with the following contents:
package csvformat
import (
"bytes"
"encoding/csv"
"fmt"
"io"
"strconv"
)
// Movie will hold our parsed CSV
type Movie struct {
Title string
Director string
Year int
}
// ReadCSV gives shows some examples of processing CSV
// that is passed in as an io.Reader
func ReadCSV(b io.Reader) ([]Movie, error) {
r := csv.NewReader(b)
// These are some optional configuration options
r.Comma = ';'
r.Comment = '-'
var movies []Movie
// grab and ignore the header for now
// we may also wanna use this for a dictionary key or
// some other form of lookup
_, err := r.Read()
if err != nil && err != io.EOF {
return nil, err
}
// loop until it's all processed
for {
record, err := r.Read()
if err == io.EOF {
break
} else if err != nil {
return nil, err
}
year, err := strconv.ParseInt(record[2], 10,
64)
if err != nil {
return nil, err
}
m := Movie{record[0], record[1], int(year)}
movies = append(movies, m)
}
return movies, nil
}
// AddMoviesFromText uses the CSV parser with a string
func AddMoviesFromText() error {
// this is an example of us taking a string, converting
// it into a buffer, and reading it
// with the csv package
in := `
- first our headers
movie title;director;year released
- then some data
Guardians of the Galaxy Vol. 2;James Gunn;2017
Star Wars: Episode VIII;Rian Johnson;2017
`
b := bytes.NewBufferString(in)
m, err := ReadCSV(b)
if err != nil {
return err
}
fmt.Printf("%#vn", m)
return nil
}
- Create a file called write_csv.go with the following contents:
package csvformat
import (
"bytes"
"encoding/csv"
"io"
"os"
)
// A Book has an Author and Title
type Book struct {
Author string
Title string
}
// Books is a named type for an array of books
type Books []Book
// ToCSV takes a set of Books and writes to an io.Writer
// it returns any errors
func (books *Books) ToCSV(w io.Writer) error {
n := csv.NewWriter(w)
err := n.Write([]string{"Author", "Title"})
if err != nil {
return err
}
for _, book := range *books {
err := n.Write([]string{book.Author,
book.Title})
if err != nil {
return err
}
}
n.Flush()
return n.Error()
}
// WriteCSVOutput initializes a set of books
// and writes the to os.Stdout
func WriteCSVOutput() error {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}
return b.ToCSV(os.Stdout)
}
// WriteCSVBuffer returns a buffer csv for
// a set of books
func WriteCSVBuffer() (*bytes.Buffer, error) {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}
w := &bytes.Buffer{}
err := b.ToCSV(w)
return w, err
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the csvformat import to use the path you set up in step 2:
package main
import (
"fmt"
"github.com/agtorre/go-cookbook/chapter1/csvformat"
)
func main() {
if err := csvformat.AddMoviesFromText(); err != nil {
panic(err)
}
if err := csvformat.WriteCSVOutput(); err != nil {
panic(err)
}
buffer, err := csvformat.WriteCSVBuffer()
if err != nil {
panic(err)
}
fmt.Println("Buffer = ", buffer.String())
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see the following output:
$ go run main.go
[]csvformat.Movie{csvformat.Movie{Title:"Guardians of the
Galaxy Vol. 2", Director:"James Gunn", Year:2017},
csvformat.Movie{Title:"Star Wars: Episode VIII", Director:"Rian
Johnson", Year:2017}}
Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
Buffer = Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
In order to explore reading a CSV format, we first represent our data as a struct. It's very useful in Go to format data as a struct, as it makes things such as marshaling and encoding relatively simple. Our read example uses movies as our data type. The function takes an io.Reader interface that holds our CSV data as an input. This could be a file or a buffer. We then use that data to create and populate a Movie struct, including converting the year into an integer. We also add options to the CSV parser to use ; as the separator and - as a comment line.
Next, we explore the same idea, but in reverse. Novels are represented with a title and an author. We initialize an array of novels and then write specific novels in the CSV format to an io.Writer interface. Once again, this can be a file, stdout, or a buffer.
The CSV package is an excellent example of why you'd want to think of data flows in Go as implementing common interfaces. It's easy to change the source and destination of our data with small one-line tweaks, and we can easily manipulate CSV data without using an excessive amount of memory or time. For example, it would be possible to read from a stream of data one record at a time and write to a separate stream in a modified format one record at a time. Doing this would not incur significant memory or processor usage.
Later, when we explore data pipelines and worker pools, you'll see how these ideas can be combined and how to handle these streams in parallel.
Working with temporary files
We've created and made use of files for a number of examples so far. We've also had to manually deal with cleanup, name collision, and more. Temporary files and directories are a quicker, simpler way to handle these cases.
Getting ready
Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/tempfiles.
- Navigate to this directory.
- Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/tempfiles, or use this as an exercise to write some of your own code!
- Create a file called temp_files.go with the following contents:
package tempfiles
import (
"fmt"
"io/ioutil"
"os"
)
// WorkWithTemp will give some basic patterns for working
// with temporary files and directories
func WorkWithTemp() error {
// If you need a temporary place to store files with
// the same name ie. template1-10.html a temp directory
// is a good way to approach it, the first argument
// being blank means it will use create the directory
// in the location returned by
// os.TempDir()
t, err := ioutil.TempDir("", "tmp")
if err != nil {
return err
}
// This will delete everything inside the temp file
// when this function exits if you want to do this
// later, be sure to return the directory name to the
// calling function
defer os.RemoveAll(t)
// the directory must exist to create the tempfile
// created. t is an *os.File object.
tf, err := ioutil.TempFile(t, "tmp")
if err != nil {
return err
}
fmt.Println(tf.Name())
// normally we'd delete the temporary file here, but
// because we're placing it in a temp directory, it
// gets cleaned up by the earlier defer
return nil
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:
package main
import "github.com/agtorre/go-cookbook/chapter1/tempfiles"
func main() {
if err := tempfiles.WorkWithTemp(); err != nil {
panic(err)
}
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see (with a different path) the following output:
$ go run main.go
/var/folders/kd/ygq5l_0d1xq1lzk_c7htft900000gn/T
/tmp764135258/tmp588787953
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
Creating temporary files and directories can be done using the ioutil package. Although you must still delete the files yourself, RemoveAll is the convention, and it will do that for you with only one extra line of code.
When writing tests, the use of temporary files is highly recommended. It's also useful for things such as build artifacts and more. The Go ioutil package will try and honor the OS preferences by default, but it allows you to fall back to other directories if required.
Working with text/template and HTML/templates
Go provides rich support for templates. It is simple to nest templates, import functions, represent variables, iterate over data, and so on. If you need something more sophisticated than a CSV writer, templates may be a great solution.
Another application for templates is for websites. When we want to render server-side data to the client, templates fit the bill nicely. At first, Go templates can appear confusing. This chapter will explore working with templates, collecting templates inside of a directory, and working with HTML templates.
Getting ready
Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.
How to do it...
These steps cover writing and running your application:
- From your terminal/console application, create a new directory called chapter1/templates.
- Navigate to this directory.
- Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/templates, or use this as an exercise to write some of your own!
- Create a file called templates.go with the following contents:
package templates
import (
"os"
"strings"
"text/template"
)
const sampleTemplate = `
This template demonstrates printing a {{ .Variable |
printf "%#v" }}.
{{if .Condition}}
If condition is set, we'll print this
{{else}}
Otherwise, we'll print this instead
{{end}}
Next we'll iterate over an array of strings:
{{range $index, $item := .Items}}
{{$index}}: {{$item}}
{{end}}
We can also easily import other functions like
strings.Split
then immediately used the array created as a result:
{{ range $index, $item := split .Words ","}}
{{$index}}: {{$item}}
{{end}}
Blocks are a way to embed templates into one another
{{ block "block_example" .}}
No Block defined!
{{end}}
{{/*
This is a way
to insert a multi-line comment
*/}}
`
const secondTemplate = `
{{ define "block_example" }}
{{.OtherVariable}}
{{end}}
`
// RunTemplate initializes a template and demonstrates a
// variety of template helper functions
func RunTemplate() error {
data := struct {
Condition bool
Variable string
Items []string
Words string
OtherVariable string
}{
Condition: true,
Variable: "variable",
Items: []string{"item1", "item2", "item3"},
Words:
"another_item1,another_item2,another_item3",
OtherVariable: "I'm defined in a second
template!",
}
funcmap := template.FuncMap{
"split": strings.Split,
}
// these can also be chained
t := template.New("example")
t = t.Funcs(funcmap)
// We could use Must instead to panic on error
// template.Must(t.Parse(sampleTemplate))
t, err := t.Parse(sampleTemplate)
if err != nil {
return err
}
// to demonstrate blocks we'll create another template
// by cloning the first template, then parsing a second
t2, err := t.Clone()
if err != nil {
return err
}
t2, err = t2.Parse(secondTemplate)
if err != nil {
return err
}
// write the template to stdout and populate it
// with data
err = t2.Execute(os.Stdout, &data)
if err != nil {
return err
}
return nil
}
- Create a file called template_files.go with the following contents:
package templates
import (
"io/ioutil"
"os"
"path/filepath"
"text/template"
)
//CreateTemplate will create a template file that contains data
func CreateTemplate(path string, data string) error {
return ioutil.WriteFile(path, []byte(data),
os.FileMode(0755))
}
// InitTemplates sets up templates from a directory
func InitTemplates() error {
tempdir, err := ioutil.TempDir("", "temp")
if err != nil {
return err
}
defer os.RemoveAll(tempdir)
err = CreateTemplate(filepath.Join(tempdir, "t1.tmpl"),
`Template 1! {{ .Var1 }}
{{ block "template2" .}} {{end}}
{{ block "template3" .}} {{end}}
`)
if err != nil {
return err
}
err = CreateTemplate(filepath.Join(tempdir, "t2.tmpl"),
`{{ define "template2"}}Template 2! {{ .Var2 }}{{end}}
`)
if err != nil {
return err
}
err = CreateTemplate(filepath.Join(tempdir, "t3.tmpl"),
`{{ define "template3"}}Template 3! {{ .Var3 }}{{end}}
`)
if err != nil {
return err
}
pattern := filepath.Join(tempdir, "*.tmpl")
// Parse glob will combine all the files that match
// glob and combine them into a single template
tmpl, err := template.ParseGlob(pattern)
if err != nil {
return err
}
// Execute can also work with a map instead
// of a struct
tmpl.Execute(os.Stdout, map[string]string{
"Var1": "Var1!!",
"Var2": "Var2!!",
"Var3": "Var3!!",
})
return nil
}
- Create a file called html_templates.go with the following content:
package templates
import (
"fmt"
"html/template"
"os"
)
// HTMLDifferences highlights some of the differences
// between html/template and text/template
func HTMLDifferences() error {
t := template.New("html")
t, err := t.Parse("<h1>Hello! {{.Name}}</h1>n")
if err != nil {
return err
}
// html/template auto-escapes unsafe operations like
// javascript injection this is contextually aware and
// will behave differently
// depending on where a variable is rendered
err = t.Execute(os.Stdout, map[string]string{"Name": " <script>alert('Can you see me?')</script>"})
if err != nil {
return err
}
// you can also manually call the escapers
fmt.Println(template.JSEscaper(`example
<example@example.com>`))
fmt.Println(template.HTMLEscaper(`example
<example@example.com>`))
fmt.Println(template.URLQueryEscaper(`example
<example@example.com>`))
return nil
}
- Create a new directory named example.
- Navigate to example.
- Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:
package main
import "github.com/agtorre/go-cookbook/chapter1/templates"
func main() {
if err := templates.RunTemplate(); err != nil {
panic(err)
}
if err := templates.InitTemplates(); err != nil {
panic(err)
}
if err := templates.HTMLDifferences(); err != nil {
panic(err)
}
}
- Run go run main.go.
- You may also run these:
go build
./example
You should see (with a different path) the following output:
$ go run main.go
This template demonstrates printing a "variable".
If condition is set, we'll print this
Next we'll iterate over an array of strings:
0: item1
1: item2
2: item3
We can also easily import other functions like strings.Split
then immediately used the array created as a result:
0: another_item1
1: another_item2
2: another_item3
Blocks are a way to embed templates into one another
I'm defined in a second template!
Template 1! Var1!!
Template 2! Var2!!
Template 3! Var3!!
<h1>Hello! <script>alert('Can you see
me?')</script></h1>
example x3Cexample@example.comx3E
example <example@example.com>
example+%3Cexample%40example.com%3E
- If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.
How it works...
Go has two template packages--text/template and html/template. These share functionality and a variety of functions. In general, use html/template to render websites and text/html for everything else. Templates are plain text, but variables and functions can be used inside of curly brace blocks.
The template packages also provide convenience methods to work with files. The example creates a number of templates in a temporary directory and then reads them all with a single line of code.
The html/template package is a wrapper around the text/template package. All of the template examples work with the html/template package directly, using no modification and only changing the import statement. HTML templates provide the added benefit of context-aware safety. This prevents things such as JavaScript injection.
The template packages provide what you'd expect out of a modern template library. It's easy to combine templates, add application logic, and ensure safety when emitting results to HTML and JavaScript.
