Grexmk: Speeding Up Scripted Builds

Slow builds can be extremely costly; for example, one of our customers loses forty percent of their developers’ productivity to waiting on builds. Build avoidance—building only what must be built to integrate a change—solves the problem but makes the build unreliable unless the build script captures all dependences of build outputs on sources. Therefore, in practice, integrating even a minor change often requires rebuilding a large software system from scratch.

Our solution can be summarized as “trust but verify”. Exploiting the fact that large software systems tend to be composed of loosely coupled parts, we semi-automatically divide a large, all-or-nothing build into many small “mini-builds”, without changing the original build. Each mini-build declares its dependences, which are trusted while scheduling mini-builds to integrate a change but verified by executing each mini-build in a sandbox in which only its declared dependences are available.

We implemented these ideas in a tool suite called Grexmk and applied Grexmk to two large all-or-nothing builds. On the first build, we demonstrated linear speedups to six build machines. On the second build, we sped up the average time for an incremental build by a factor of 1.2. the build script captures all dependences of output files on source files. Therefore, in practice, integrating even a minor change often requires rebuilding a large software application from scratch.