Build system comparison: Mage vs bob

Introduction

In my first article where I introduced you to bob, I made you a promise that I will try and see if bob can replace Mage, the build system used by the Hugo team.

The short answer is yes, and in this post, I will show you what I learned in my journey to switch from mage to bob.

As a disclaimer, I’m currently working at benchkram, the company behind bob, so I’ll try to be as less biased as possible. I’m still new to both bob and Mage, so I hope that my fresh eyes on both projects will make this an interesting post.

Before getting started

On Hugo's contribution page you will find the instructions on how to install Mage and clone Hugo locally. Every command I describe here is relative to the cloned hugo directory.

Comparing Magefile with the Bobfile

Here is the link to the Magefile. The corresponding Bobfile which should do the same things is looking like this:

Place this bob.yaml file in the same working directory as you can run both bob and mage without conflicts.

With everything in place, let’s see what I observed:

Install experience

Since the Magefile is just a Go file, the only dependency you have to install for Mage is the Go standard library. Bob is a single binary that doesn’t require any other dependency except Nix which is used to install your project dependencies.

Platform support

One of the first things I notice is that Mage has Windows support while bob has only support for Linux and OS X.

Since bob is written in Go, I guess future support for Windows is technically possible. For now, the bob docs state that “we do not take any efforts to make it Windows compatible right now.”

User-friendly commands

I really like the less verbose commands for Mage like: mage hugo, mage install, mage uninstall.

For the same targets created with bob, you would have to type bob build, bob build install, bob build uninstall. This is because bob, being more than a task runner, is offering more commands such as bob run, bob auth,or bob git.

I would drop the build and run words from bob commands. Maybe I'm being lazy, but I like when I achieve more with less typing.

Verbosity of commands

Speaking of user-friendly commands, Mage has a -v option to show you a nice verbose output when running targets. For example, when running mage -v hugo you get:

Running target: Hugo
exec: git "rev-parse" "--short" "HEAD"
exec: go "build" "-ldflags" "-X github.com/gohugoio/hugo/common/hugo.vendorInfo=mage" "-tags" "none" "github.com/gohugoio/hugo"

bob also has -v flag, which you can use to see what bob does under the hood, but would be nice to see all exec calls similar to Mage.

Description of targets

Since Magefile is just a plain old go file and the targets are Go functions, you can add comments to them and know later what each target is doing:

And it’s pretty neat that running mage on our Hugo repo will output all those targets with their descriptions:

$ mage
Targets:
  check            Run tests and linters
  docker           Build hugo Docker container todo
  fmt              Run gofmt linter
  genDocsHelper    Generate docs helper todo looks like this doesnt work
  generate         autogen packages todo looks like this doesnt work
  hugo             Build hugo binary
  hugoNoGitInfo    Build hugo without git info
  hugoRace         Build hugo binary with race detector enabled
  install          hugo binary
  lint             Run golint linter
  test             Run tests
  test386          Run tests in 32-bit mode Note that we don't run with the extended tag.
  testCoverHTML    Generate test coverage report
  testRace         Run tests with race detector
  uninstall        hugo binary
  vet              Run go vet linter

bob doesn’t provide this functionality, even though it comes with an autocompletion feature for the main commands. Would be nice if you could add a description node to each task where a developer can add what that task is doing.

Dependency management

If you follow the Hugo contribution guide you think that you only need to install Mage to work with Hugo. However, when I run mage lint, I get the following error:

Running target: Lint
exec: go "list" "./..."
exec: golint "."
ERROR: running go lint on ".": failed to run "golint .: exec: "golint": executable file not found in $PATH"

Looks like you need golint installed on your system to run the lint target. You have no other choice but to manually installgolint.

The nice thing with bob is that you can define your task’s dependencies in your bobfile. This way you never lose track of what dependencies are needed for your builds to work.

I wrote how bob can use Nix to manage project dependencies, but for our post here’s how we define the lint task:

Now we don’t need to worry if we don’t have golint installed on our system. Nix will fetch it for us.

bob is language agnostic

The fact that Mage is a Go file gives you the freedom to write more complex logic in your targets. But this also comes with a downside: it is appealing only to Go developers.

bob tries to avoid that: even though is a tool written in Go, it is language agnostic using just a plain .yaml file to describe your build pipeline.

If you don’t know Go it might be harder to understand the Magefile, while bob will be pretty familiar to everyday CLI users. You could even run Mage commands in the Bobfile for that matter.

Environment variables

With Mage, you can add on the fly whatever environment variables you need for your commands:

func Test386() error {
	env := map[string]string{"GOARCH": "386", "GOFLAGS": testGoFlags()}
	return runCmd(env, goexe, "test", "./...")
}

With bob, you can only do that per bobfile with the variables node which will add those environment variables to every command:

which means that if you want custom environment variables you would need to use export on the cmd node:

Would be nice to have a way to declare the environment variables for a specific task in bob.

Case-insensitive commands

I like that Mage commands are case-insensitive. For HugoRace(), I can run mage HugoRace or mage hugorace.

If I do something similar with bob I get a task does not exist error. I think that case of the commands should not matter. If you happen to have 2 tasks with the same name but a different case, then you probably need to think of a better name.

Pinning a dependency version

In Fmt() target, Mage is checking for isGoLatest() and if it's not installed, it will just stop making the user think that everything went ok:

There are 2 problems with this: First, if I’m running go1.18.4(which currently is indeed the latest version) this task will stop without any warning making me think everything went fine. Second, whoever manages this Magefile will always have to update it every time the Go version changes.

When creating a reproducible build you cannot leave dependencies to the user. Everyone has distinct packages installed on their system and if you want the same build to work for everyone, you need to take control of the dependencies.

bob goes in this direction by introducing reproducibility with Nix (the build is independent of the local environment) allowing you to pin an exact version of a package you need.

Combining multiple commands

Often times you want to run multiple commands in a certain order. You can run multiple targets on Mage with mage foo bar baz or just have a custom target as they do with Check where they run Fmt, Vet and TestRace:

You can also achieve that with bob declaring:

Running with Docker

For some reason, running mage docker failed with the following error:

shell Error: No such container: hugo-build docker: invalid reference format. See 'docker run --help'. Error: running "docker run --name hugo-build hugo ls /go/bin" failed with exit code 125

In a future article, I will cover how bob plays with docker, but as long as you build the binary with bob/Mage, you can run it with a simple Dockerfile:

FROM debian:buster-slim
COPY hugo /usr/bin/hugo
# libc6-compat & libstdc++ are required for extended SASS libraries
# ca-certificates are required to fetch outside resources (like Twitter oEmbeds)
RUN apt update && \
    apt install -y  ca-certificates libc6
VOLUME /site
WORKDIR /site
# Expose port for live server
EXPOSE 1313
ENTRYPOINT ["hugo"]
CMD ["--help"]

Conclusion

That was it! I hope you enjoyed this post and got a better feeling of working with Mage or bob. build systems come in different flavors so it’s good to understand their differences and make the right choice for your use case.

I think both Mage and bob are both great tools for their indented audiences. No matter which one you choose, don’t forget to contribute to other open-source. Hugo is awesome!