GO EP12: Go.mod File - Boring Stuff Made Easy

There are several directives we will talk about, including: module, go, require, exclude, replace, retract, and a new one, toolchain."

Photo by CHUTTERSNAP on Unsplash

In Go, the go.mod file is all about managing modules. We’ve talked about this a lot before, but let’s dive deeper this time

The go.mod file is set up so that each line has a specific job, called a directive. Each directive starts with a keyword that tells Go what to do, followed by the needed details or values.

For instance, a line in the go.mod file might declare the name of the module, a dependency on another module, the version of your module, etc:

module github.com/user/project

go 1.18
require (
    example.com/othermodule v1.2.3
    github.com/user/otherproject/v2 v2.2.3
)g

There are several directives we will talk about, including: module, go, require, exclude, replace, retract, and a new one, toolchain."

Module directive

A module is a collection of Go packages grouped together, and it needs a special name called a “module path.” This is a unique identifier that tells others where to find your module.

module github.com/user/project-a
module github.com/user/project-a/v2

Here, github.com/user/project-a is the module path. If your module is version 2 or higher, you have to add the version number at the end of the module path.

“Why does the module path always look like a URL?”

It usually looks like a URL (a web address) because it’s often linked to where your code is stored online, like on GitHub or GitLab. This helps Go tools easily find and download your module.

But if your module isn’t meant to be used as a library, you can choose any module path that’s unique. Instead of using a real URL where the code can be found, you just use a name or path that you control and that won’t conflict with others.

module helloworld

One cool thing is that you can deprecate, or gradually phase out, a module with a simple line of text:

// Deprecated: Use github.com/user/project-a/v2 instead.
module github.com/user/project-a

This deprecation note applies to all minor updates of the module. For instance, if github.com/user/project-a includes versions like v1.0.1, v1.2.0, and so on, the deprecation message covers all these versions at once.

So, if you create a new major version with a different path, like github.com/user/project-a/v2, it’s treated as a separate module, and the deprecation of version 1 doesn’t automatically affect it.

When tools like go get run, they will notice this deprecation message and warn users accordingly.

“But I just want to deprecate a specific version of my module, as I made a mistake”

In this case, you can use the retract directive, it lets you specify a version or a range of versions that have problems and shouldn’t be used. We'll go into more detail about this soon.

Go directive

The go directive tells everyone which minimum version of Go they need to have installed to build and run your module successfully.

go 1.18

require (
    github.com/user/dependency v1.2.3
)

While it suggests the minimum Go version needed, the Go toolchain doesn’t strictly enforce it (until Go 1.21). You can use a lower version of Go to build your module as long as you don’t use any new features introduced after that lower version.

Let’s explain with a couple of scenarios based on Go versions below 1.21:

• If you use number underscore literals like 1_000_000 in a module that has a go 1.12 directive, you’ll get a compile error because this feature was introduced in Go 1.13.

• If you use those literals in a module with a go 1.13 directive, but your machine has Go 1.12, you’ll still get an error.

• If you don’t use those literals, and the directive is go 1.13, but your machine has Go 1.12, you can still build the module.

// go.mod file
go 1.12

package main

// compile error: underscores in numeric literals requires go1.13 or later
var a = 1_000_000

“What if my project uses Go 1.12, but one of my dependencies requires Go 1.13 and uses 1_000_000?"

The logic stays the same.

If you’re using 1_000_000 in your main module which is set for Go 1.12, no doubt, you’ll need to upgrade your main module to at least Go 1.13. But if one of your dependencies uses this new features, then it depends on the Go version on your machine. If your machine’s Go version is lower than what’s required, you’ll get an error.

Again, this is assuming you’re using Go 1.20 or below.

“So does Go 1.21 change the mechanism?”

Yes, starting with Go 1.21, things got stricter. Now, the go directive is a hard requirement. If a module specifies a Go version newer than the one installed on your system, the toolchain will refuse to use that module.

Basically, your main module’s go version must be at least as high as any of your dependencies’ go versions. So, the problems we talked about with Go 1.12 and Go 1.13 don’t happen in Go 1.21 and later.

Toolchain Directive

The go directive makes sure the code works with a specified version of Go, but it doesn’t let you pick a preferred or exact version of the toolchain to use.

“Wait, what is a toolchain?”

In Go, a toolchain is just a set of tools that help you run, build, and test your project. This includes stuff like the Go compiler, Go assembler, Go linker, standard libraries like fmt and os, etc. We’ll discuss more about the Go toolchain in Go 1.21 when we dive into it (I’ll try to remember to add a link here).

As Go evolves, you might need different toolchain versions for different projects.

module example.com/mymodule

go 1.21
toolchain go1.21.3

This means that while the module needs at least Go 1.21, it prefers to use the Go 1.21.3 toolchain if it’s available.

Require directive

When you use the require directive, you’re setting the minimum version of a module that your code needs.

require (
    github.com/user/project-a v1.2.3
    github.com/user/project-b v1.5
    github.com/user/project-c v1
)

The first line means your module needs at least version v1.2.3 of github.com/user/project-a. The Go tool will make sure to use this version or a newer one if any other modules require it.

But the second and third lines might look a bit odd. You probably haven’t seen this often, and it works a bit differently.

The second line is saying we need the highest minor version of v1.5 for github.com/user/project-b. But this doesn’t last long, after running Go commands to update go.mod, this line will be replaced with the exact version, like v1.5.99 if 99 is the highest minor version available.

Similarly, for the third line, it will find the latest version of v1 for github.com/user/project-c and update the version to that exact number.

Direct & indirect

Direct dependencies are the ones your module directly imports and uses in its code.

Indirect dependencies are those needed by your direct dependencies but aren’t directly used by your module. In these cases, the go.mod file might include require directives marked with // indirect.

require (
    github.com/user/project-a v1.2.3
)

require (
    github.com/user/project-b v1.5.0 // indirect
)

Here, your main module (the project you’re working on) directly uses project-a, but project-a needs project-b, and your module doesn’t use project-b directly. So, project-b is listed as an indirect dependency, and it’s marked with // indirect.

Note that the mechanism can differ in Go 1.16 and below.

In those versions, indirect dependencies are added to go.mod only when a higher version is picked through upgrades (go get) or when other dependencies are removed (go mod tidy). Starting from Go 1.17 and above, the Go tool always adds indirect dependencies for any module that provides packages used by your module or its dependencies.

Replace Directive

Imagine you need to use a different version of a dependency than what’s normally used.

Maybe you’re testing changes or you need a newer version that isn’t officially released yet. The replace directive lets you tell Go to use your chosen version instead of the default one.

replace (
    example.com/project-a v1.2.3 => example.com/project-a v1.2.4
    example.com/project-b => example.com/project-b v1.5.1
    example.com/project-c => ./submodule/project-c
)

There are a bunch of options here.

You can replace a specific version with another specific version, replace an entire version range with a specific version, replace an entire version with a local directory, or even replace a specific version with a local directory.

require (
    example.com/project-a v1.2.4
)

replace example.com/project-a v1.2.4 => example.com/project-a v1.2.3

In this scenario, we originally use project-a@v1.2.4, but if we find issues when upgrading to this version, we can quickly test things by switching back to project-a@v1.2.3 using the replace directive to check compatibility.

Even though you won’t see a require directive for project-a version v1.2.3 in your go.mod file, whether directly or indirectly, the Go tool or Go commands are smart enough to use the version you’ve specified with replace. If you run go mod tidy now, Go will download v1.2.3 into the module cache (usually $GOPATH/pkg/mod), update go.mod, and adjust go.sum to reflect its dependencies.

This change updates your module graph as if you’re using version v1.2.3, as if v1.2.4 doesn’t exist. We covered the module graph in previous discussion.

Remember, the replace directive only changes how dependencies are resolved within your main project. If your project is a dependency for others, the main module won't see or use the replace directive from your go.mod file.

“Is it useful for production?”

Even though it’s rare, the replace directive has its uses and can still meet certain needs, especially when pointing to a forked repo or using a monorepo with local modules.

We’ve actually used this as a workaround when our multiple services shared the same models, but we didn’t want to duplicate the code in every repository.

We handled this by using Git submodules along with the replace directive to make it work like a monorepo but with Git versioning for each module. It’s pretty handy and convenient, at least until we overused it.

Things got messy when we didn’t manage or control it properly, and our submodules got bloated with a bunch of shared code. Sharing without clear ownership or responsibility can turn into a headache.

Exclude Directive

If a specific version of a dependency is buggy, has issues, or just doesn’t work with your project, you can use the exclude directive to handle it.

exclude (
    github.com/user/project-a v0.4.0
)

By excluding these versions, we make sure they won’t be used, even if other dependencies in your project try to require them.

After adding the exclude and running go mod tidy, the Go tool will look for higher versions and use those, assuming those higher versions aren’t excluded.

$ go mod tidy

go: dropping requirement on excluded version github.com/user/project-a v0.4.0
go: finding module for package github.com/user/project-a
go: found github.com/user/project-a in github.com/user/project-a v0.6.0

So, you’re telling Go, “Hey, don’t use v0.4.0 of project-a, find a better version,” and it’ll do just that.

“What if I exclude all versions from v0.4.0 to the latest?”

In this case, running go mod tidy will downgrade to v0.3.0, at least that’s what I found in my research with Go 1.22.3.

It’s quite surprising because modules or libraries are built to be backward compatible, not forward compatible (older versions are supposed to work with future, newer versions). So instead of throwing an error, Go will fallback to the older version, even if it’s lower than what’s typically required.

Retract Directive (Go 1.16)

When you realize a specific version of your module has a serious problem or was released by mistake, you can use the retract directive to tell others not to depend on that version anymore.

To retract a previous version, we add a retract line, specify the version or range of versions to avoid, and then create a higher version with the retraction directive.

For example, if you released version v1.2.0 of your module too soon and it had major issues, you’d add a line like this in your go.mod:

retract v1.2.0 // Typo in the version

// or a range, retract from v1.1.0 to v1.2.0
retract [v1.1.0, v1.2.0]

Be careful when you retract a version like v1.2.0. 

If you find the error and want to immediately notify users but plan to fix it later, you should also retract the version that only includes the retraction notice, like v1.2.1, to make sure users don’t accidentally use v1.2.1 either.

retract (
    v1.2.0 // Typo in the version
    v1.2.1 // Contains the retraction directive
)

The Go commands will ignore retracted versions, but they won’t throw an error if you’re still using them. go get will skip retracted versions and choose another version to download or upgrade to unless you or your module directly specify the retracted version.

Of course, The retract directive is just a way to mark a version or range of versions as flawed without removing them. Our module still appears in version control or on servers that host it, so any existing builds relying on that version won’t break.