Dependency Management

"If at first you don't succeed, call it version 1.0."

- Pat Rice

Being strong believers in the SOLID principles we discussed in the previous chapter, several prominent figures in the Go community strongly advise software engineers to organize their code into self-contained and reusable packages.

When our code imports an external package, its dependency graph is augmented not only with the imported package but also with its set of transitive dependencies—that is, any other packages (and their dependencies) required by the packages that we import. As our projects grow larger in size, it becomes necessary to efficiently manage the versions of all our dependencies to ensure that changes in upstream transitive dependencies do not cause unexpected side effects (crashes, changes in behavior, and so on) to our own programs.

In this chapter,...

What's all the fuss about software versioning?

The idea of versioning is ingrained into everything around us. People all over the world are accustomed to using various forms of versioning on a daily basis. Note that I am not just talking about software here. The vast majority of physical products that you are using are associated with some sort of versioning scheme. Uses of versioning range from your computer's CPU to your mobile phone, and from the revision of the algorithm book on your bookshelf to your favorite superhero (or alternatively light-saber-wielding rebel) movie.

When we move to the realm of software, the concept of versioning becomes even more important. Nowadays, as more and more software engineers ascribe to the release fast mantra, having a sane versioning system in place makes it possible to do the following:

• Validate that a particular piece of software...

Vendoring – the good, the bad, and the ugly

The fact that services such as gopkg.in always redirect the go get tool to the latest available major version for a given version selector is, technically speaking, a show-stopper for engineering teams that endeavor to set up a development pipeline that guarantees repeatable builds. The typical CI pipeline will always pull both compile and test dependencies via a command such as go get -t -u ... prior to building the final output artifact. As a result, even if your code has not changed between builds, your service or application binary may be different because of changes in the dependencies that get pulled in.

However, what if I told you that there is actually a way to retain the benefits of lazy package resolution and at the same time have the flexibility to pin down package versions for each build? The mechanism that will assist...

Strategies and tools for vendoring dependencies

Initially, Go had no support for vendoring packages. This made sense at the time, as Google, the primary user of Go, would host all of their package dependencies in a single repository (commonly referred to as a mono-repo).

However, as the Go community began growing and more and more companies began porting their code bases to Go, dependency management became an issue. With the release of Go 1.5, the Go team added experimental support for vendoring folders. Users could enable this feature by defining an environment variable named GO15VENDOREXPERIMENT.

When this feature is enabled, each time the Go compiler attempts to resolve an import, it will first check whether the imported package exists inside the vendor folder and use it if found; otherwise, it will proceed, as usual, to scan each entry in the $GOPATH looking for the package...

Summary

In this chapter, we discussed the reasons that necessitate the use of versioning for not only the packages that our code imports, but also the code itself that we, as software engineers, author. We then defined the concept of semantic versioning and the circumstances where each component of a semantic version needs to be incremented.

The meat of the chapter dealt with the concepts of vendoring as the primary mechanism for ensuring repeatable builds for our projects. After elaborating on the pros and cons of vendoring as a process, we examined the current state of vendoring in the Go ecosystem and provided a brief tour of the state-of-the-art tools (dep and Go modules) that engineers should use to manage their package dependencies.

Of course, as our code base evolves and the version requirements for our imports change over time, it is likely that, at some point, a newer...

Questions

1. Why is software versioning important?
2. What does a semantic version look like and when are its individual components incremented?
3. Which component of a package's semantic version would you increment in the following cases?
   1. A new API is introduced.
   2. An existing API is modified and a new, required parameter is added to it.
   3. A fix for a security bug is committed.
4. Name some alternative versioning schemes that we could use besides semantic versioning.
5. What are the pros and cons of vendoring?
6. Name some of the differences between the dep tool and Go modules.

Further reading

1. Cook, Stephen A., 'The Complexity of Theorem-proving Procedures', Proceedings of the Third Annual ACM Symposium on Theory of Computing, STOC '71. New York, NY, USA, ACM, 1971, S. 151–8
2. dep: a dependency management tool for Go: https://github.com/golang/dep
3. glide: https://github.com/Masterminds/glide
4. gvt: https://github.com/FiloSottile/gvt
5. godep: https://github.com/tools/godep
6. Golang blog: using Go modules: https://blog.golang.org/using-go-modules
7. Gopkg.in: stable APIs for the Go language: https://labix.org/gopkg.in
8. Gopkg.toml format specification: https://golang.github.io/dep/docs/Gopkg.toml.html
9. govendor: https://github.com/kardianos/govendor
10. Plus codes: short codes for locations, for places that don't have their own street address: https://plus.codes/
11. Semantic versioning 2.0.0: https://semver.org/
12. Silva, João P. Marques; Lynce...