Redo implementation in Bourne Shell

I have implemented the build system redo as designed by DJB in Bourne Shell. To understand how redo can be simpler, more flexible, more powerful and more reliable than make, read Introduction to redo and/or redo: a top-down software build system. The current version of redo is redo 2.0.0.

redo
the main program
redo-always
marks the current target as always needing to be rebuilt
redo-dot
prints redo dependency graph in DOT format (example output, example rendering)
redo-ifchange
adds dependencies for the current target (if a dependency changes, the target will be rebuilt)
redo-ifcreate
adds non-existence dependencies for the current target (if a non-existence dependency is created, the target will be rebuilt)
redo-ood
prints a list of all target files that are out of date
redo-targets
prints a list of all target files that exist (a target is a file redo can build)
redo-sources
prints a list of all source files that exist (a source is a dependency that is not a target)
redo-stamp
detects if the current target has changed (see apenwarr's documentation)

Installation

To install redo, copy the redo executables and man pages to a directory in your $PATH. For example, if /usr/local/bin exists, is listed in your $PATH and can be written to by the current user, the following commands will install redo:

wget http://news.dieweltistgarnichtso.net/bin/archives/redo-sh.tar.gz
tar -oxzf redo-sh.tar.gz -C /usr/local

This implementation of redo depends only on GNU Core Utilities or BusyBox.

Frequently Asked Questions

How do I build a file with redo?

To build a file target, you have to create a dofile target.do with shell commands that write the intended result of the build either to standard output or to its parameter $3. Then run redo target.

The default target is all: redo without arguments executes commands from all.do.

What do the redo parameters $1, $2, $3 mean?

When redo runs a dofile, it gives it three parameters:

$1filename of target
$2basename of target, without extension if default dofile is used
$3filename of temporary output file that is renamed on build success

How does the extension removal in parameter $2 work?

Redo removes the extension that it gets from the default dofile filename:

TargetDofileParameter $2
a.b.ca.b.c.doa.b.c
a.b.cdefault.doa.b.c
a.b.cdefault.c.doa.b
a.b.cdefault.b.c.doa

How do I declare dependencies with redo?

Use the redo-ifchange command in a dofile: redo-ifchange dependency inside target.do means: If the target is built, the dependency is built if it does not exist and recorded as a dependency. On subsequent builds, if dependency does not exist or has changed since the last build, both dependency and target are rebuilt.

How does this redo implementation check dependencies?

For dependency checking, this implementation of redo checks the dependencies' ctime against the stored ctime. If the ctime differs, it checks the dependencies' md5sum against the stored md5sum. This is arguably more useful than just using ctime.

How do I declare non-existence dependencies with redo?

Use the redo-ifcreate command in a dofile: redo-ifcreate ne_dependency inside target.do means: If target is built, the non-existing file ne_dependency is recorded as a non-existence dependency. If ne_dependency exists on subsequent builds, target is rebuilt.

What are command line options for this redo implementation?

Short optionLong optionEffect
-d--debugprint dependency checks as they happen
-h--helpprint usage instructions and exit
--versionprint version information and exit
-x--xtraceprint commands as they are executed (variables expanded)

How can I use this redo implementation to build in parallel?

You can use the shell builtins & and wait to execute commands asynchronously. The following line in a dofile builds targets a and b in parallel: redo-ifchange a & redo-ifchange b & ; wait

How can I make redo build everything in a different directory?

You can use union mounts to combine multiple directories. For example, on Linux with unionfs-fuse, the command unionfs -o cow 'output=RW:source=RO' /tmp/build combines the output and source directories so that /tmp/build contains all files from the source directory, but any newly created file will show up only in the output directory.

Why can a target be built twice during a run?

Some build processes require several builds of the same target. Naïve implementors of build systems disregard this possibility and do not check the validity of a target again if it was built during a run.

How does this implementation of redo compare with other implementations?

The following table compares redo implementations. Differences from the DJB redo design and incompatibilities with existing dofiles are emphasized.

Implementation Programming Language Modification Time Check File Content Hash Check Parallel Build Notes, Bugs, and Incompatibilities
Nils Dagsson Moskopp's redo Bourne Shell ✓ Yes ✓ Yes ✕ No
  • few dependencies
Alan Grosskurth's redo Bourne Shell ✕ No ✓ Yes ✕ No
  • few dependencies
  • dofiles must be shell scripts
Avery Pennarun's do Bourne Shell ✕ No ✕ No ✕ No
  • few dependencies
  • each target is always rebuilt
Avery Pennarun's redo Python ✓ Yes ✕ No ✓ Yes
  • targets are not rebuilt if modified
Christian Neukirchen's redo C ✓ Yes ✓ Yes ✓ Yes
  • standard output is not written to target
Jonathan de Boyne Pollard's redo Bourne Again Shell ✓ Yes ✓ Yes ✕ No
  • dofiles must be executable and begin with #!
  • special files are never hashed
Gyepi Sam's redux Go ✓ Yes ✓ Yes ✕ No
  • wrong handling of $2 parameter
jekor's redo Haskell ✕ No ✓ Yes ✕ No
  • $1 parameter is not provided
  • no redo-ifcreate implementation
Jonathan de Boyne Pollard's redo C++ ✓ Yes ✓ Yes ✓ Yes
  • special files are never hashed
Shanti Bouchez-Mongardé's redo Python ✓ Yes ✕ No ✓ Yes
  • dofiles are also looked for in do/ directory
  • standard output is not written to target
  • targets are not rebuilt if modified
Tharre's redo C ✕ No ✓ Yes ✕ No
  • targets are only built once