Where My Tools Go

Development Dependencies in Go

Sat, Aug 15, 2020 3-minute read

Source: Todd Quackenbush (https://unsplash.com/photos/IClZBVw5W5A)

This posts seeks to answer the question: how are development dependencies managed in Go projects?

This naturally brings up several other questions:

Is there an official way? Yes, via tools.go
How do we use tools.go? Simple, import the packages to _
What exactly does tools.go do? Well, read on

Background

In many other programming languages, part of dependencies management includes management of development dependencies—in short, things that are required for developers, but not for end-users.

In Python, when installing dependencies, if you run pipenv install --dev <package-name>, the packages will be installed as development dependencies.
In Javascript, you run npm install <package-name> --save-dev to achieve the same.
In Java, Gradle has the concept of dependency configurations, where you can configure certain dependencies such as compileOnly that are only required at compile time and not at runtime, runtimeOnly which is the reversed.
- Sidenote: sbt also supports similar concepts: see Library Dependencies and Library Management.
In C# / .NET (and .NET Core), dependencies are managed by NuGet, which supports PrivateAssets (see documenation on Controlling dependeny assets.
- Sidenote: there was an issue (now closed) regarding a DevelopmentDependency metadata, I didn't go down that rabbit hole and research what happened to it.

And that briefly covers all the "modern" (by my definition) languages that I know of or have used.

What about Go?

Official Way

As stated on the GitHub wiki on How can I track tool dependencies for a module The officially recommended way is to create a file called tools.go that imports the required development dependencies to _

As further explained by Russ Cox (one of Go's core developer), this approach doesn't introduce new mechanisms, but merely reuses existing ones.

How to Use `tools.go`

Make sure the project is using Go modules.
Create a file called tools.go at module root directly as follows:
```
  // +build tools

  package tools

  import (
      _ "golang.org/x/tools/cmd/stringer" // This is the dev dependency
  )
```
Notice the // +build tools at the beginning of the file. This ensures that the file is not compiled into the binary (I'm using loose language here, not sure if "compile" is the correct word to use).
Run go mod tidy to install the dependencies (including development dependencies)
- Sidenote: One thing that tripped me up is that go mod tidy doesn't seem to be a command to be used for installing dependencies. However, in the official Migrating to Go Modules guide, it is mentioned that "It is always good practice to run go mod tidy before committing a go.mod file to version control".
- Sidenote 2: You might think go build, go run or even go test might be the commands that'll trigger installing of development dependencies. However, the current recommended approach leverages on the fact that we are using a build constrain (with // +build tools) that will not be satisfied on any platform to prevint development dependencies from being compiled into the binary (again, I'm using loose language here).
You would notice that go.mod has been updated to contain the development dependencies.

Implications

If using go generate and vendoring, the comment should refer to the vendored binary, as follows (continuing the example above):
```
  //go:generate go run golang.org/x/tools/cmd/string
  ...
```

Note:

The examples are borrowed from: Tools and dependencies.
The note on implication above are not tested, please refer to the GitHub issues (here and here) for details.

What Exactly does `tools.go` do?

As alluded to in some of the sidenotes above, tools.go works briefly as follows:

tools.go is just like any other source file in go. As such, by importing development dependencies makes the Go suite of tools treat such development dependencies like any other dependencies.
The ingenuity comes in using the build constraint (// +build tools) to prevent the tools.go file from being actually included in the package. For more details, see official documentation on Build constraints.

I'm Yong Jie