In Praise of the ./go Script - Part I

My step-dad is a cabinet-maker by trade. We chat about his work from time to time. I've often been struck by the similarities between building furniture and building software. Take tooling for example. Choosing high quality tools and learning how to use them correctly is of course a very important part of woodworking. But besides the standard general-purpose woodworking tools - saws, lathes, planes and the like - it's very common for a cabinet-maker to spend several days building a custom tool or jig, the sole purpose of which is to enable them to do work on part of the piece they're building with greater quality or accuracy. 

ThoughtWorks teams have a similar focus on custom tooling to enhance productivity. In one recent example I was working with a very smart tech lead who spent a solid day building a custom visualization tool so that the team could get a clearer sense of how money flowed through various accounts in a trading system. That tool will never be used itself in a production setting, but it brought a lot of benefit in how the team builds production code.

I'd argue that any self-respecting dev team should pay attention to automate their processes and conventions with an array of tooling, be it off-the-shelf, customized and fully bespoke. This is where a lot of teams stop, which is a shame. It's great to have the tooling you need, but if you don't organize your workshop so that everything is at hand you're not going to be as effective as you could be. Teams can get a lot more out of their automation by unifying all of their tools and processes under a common interface - the ./go script.

As software developers we regularly create or customize "non-production" tooling for tasks like building, optimizing, profiling, migrating, and deploying our code. Building bespoke toolchains tailored for a specific team's work is a very valuable activity. This is particularly true when these toolchains are used to automate an otherwise manual process. Alarm bells are immediately ringing for me whenever I see people on a team manually doing things like migrating data or moving files into the correct locations for a build process to consume. Automating these tasks is always beneficial. An automated process is usually a time-saver, but more importantly an automated process is consistently applied. That means consistently across dev environments - no one likes to hear "it works on my machine" - but also hopefully consistently in staging, pre-prod and prod environments too.

Beyond the standard custom tooling that most teams use, every ThoughtWorks project I've been involved with in the last few years has had a ./go script of some sort. This is a script checked in to the root of a project repo which acts as the unified interface into any non-production tooling used by the team. It tends to start life as a simple wrapper around a build tool like ant, maven, rake, make. In today's polyglot world it often quickly becomes a wrapper over several tools for different parts of the stack - gradle for the backend and grunt for the frontend, perhaps. Over time the script grows in capabilities. A mature ./go script will do things like pre-flight checks for required software on your system (including checking for the correct version), fetching and installing if necessary. It will act as a front-end for any task a developer needs to do - for example start a local server, run db migrations, or run a pre-commit quality check before pushing code. I've seen ./go scripts which use vagrant to run all tooling in a standardized VM instance - something I'm a big fan of. I've even seen a ./go script which instrumented compilation times and reported them to a centralized system (along with stats on the host machine's CPU, RAM, etc). 

The ./go script becomes the only place anyone ever needs to look when they want to accomplish some dev task, meaning team members no longer need to remember things like which task is in rake, which is a rails command, and which is a ruby script. It acts as a sort of grounding rod for tribal knowledge, capturing knowledge about tools and processes in software rather than allowing it to pool inside team member's heads or "documented" in a seldom updated and rarely accurate wiki page somewhere. As a core part of the developer's day-to-day these scripts stay current - you can't get work done otherwise. If a tool changes or a new dev process is created it will be very quickly captured within the ./go script. There's no need to remind everyone at standup that we need to be sure to run `gradle foo:blah` before standing up the server locally if that process has already been baked into the one interface everyone uses.

Working on software systems usually means running local versions of those systems. This tends to be something that's cumbersome to setup and fiddly to maintain. It's common for a new team member to spend several days learning how to get their development workstation set up just right. It's also common for changes to a software system under development to require everyone on the team to manually do some task - "don't forget to run the DB migration on your local instances, everyone!". High-functioning development teams realize the benefit of aggressively automating these tasks. People can join a team and become effective almost straight away. 

You know you're on a mature dev team when your instructions as a new team member are "check out the repo, run ./go, and you're done". This will ensure any required tooling dependencies are set up and then present you with a list of sub-commands; your first introduction to what’s available in your new team’s toolchain via ./go. Any onboarding process more complex than this is an opportunity for improvement.