Yet another Make?
Many people are not satisfied with Make, otherwise there would not be dozens of other build systems and dozens of dialects of Make alone. A redo this one yet another alternative? On the one hand, of course, yes - only extremely simple, but capable of solving absolutely all the same tasks as Make. On the other hand, do we have some common and uniform Make?
Most "alternative" build systems were born because they lacked the native Make capabilities, lacked flexibility. Many systems are only concerned with generating Makefiles, not building them themselves. Many are tailored to the ecosystem of certain programming languages.
Below I will try to show that redo is a much more noteworthy system, not just another solution.
Make is always there anyway
Personally, I still always looked askance at this whole alternative, because it is either more complex, or ecosystem / language-specific, or is an additional dependency that needs to be set and learned how to use it. And Make is such a thing that, plus or minus, everyone is familiar with and knows how to use at a basic level. Therefore, always and everywhere I tried to use POSIX Make, assuming that this is something that in any case everyone has in the (POSIX) system out of the box, such as the C compiler. And the tasks in Make to perform only for which it is intended: parallelized execution of goals ) taking into account the dependencies between them.
What's the problem with just writing in Make and making sure it works on any system? After all, you can (must!) Write in the POSIX shell and not force users to install some monstrous huge GNU Bash. The only problem is that only POSIX Make dialect will work, which is scarce enough even for many small simple projects. Make on modern BSD systems is more complex and feature-full. Well, with GNU Make, few can compare with anyone, although almost no one uses its capabilities to the fullest and does not know how to use them. But GNU Make does not support a dialect of modern BSD systems. BSD systems do not have GNU Make in them (and they are understandable!).
Using a BSD / GNU dialect means potentially forcing the user to install additional software that does not come out of the box anyway. In this case, the potential advantage of Make - its presence in the system - is nullified.
It is possible to use and write in POSIX Make, but difficult. Personally, I immediately recall two very annoying cases:
- Some Make implementations, when executing $ (MAKE) -C, "go" to the directory where the new Make is executed, and some do not. Is it possible to write a Makefile so that it works the same everywhere? Of course:
tgt: (cd subdir ; $(MAKE) -C ...)
Conveniently? Definitely not. And it is unpleasant that one must constantly remember about such trifles. - In POSIX Make, there is no statement that executes a shell call and stores its result in a variable. In GNU Make up to 4.x version you can do:
VAR = $(shell cat VERSION)
and starting with 4.x, as well as in BSD dialects, you can do:
VAR != cat VERSION
Not quite the same action can be done:
VAR = `cat VERSION`
but it literally substitutes this expression in your shell commands described in the targets. This approach is used in suckless projects, but it is, of course, a crutch.
Personally, in such places, I often wrote Makefiles for three dialects at once (GNU, BSD and POSIX):
$ cat BSDmakefile
GOPATH != pwd
VERSION != cat VERSION
include common.mk
$ cat GNUmakefile
GOPATH = $(shell pwd)
VERSION = $(shell cat VERSION)
include common.mk
Conveniently? Far from it! Although the tasks are extremely simple and common. So it turns out that either:
- Write in parallel for multiple Make dialects. Trading developer time for user convenience.
- Keeping in mind a lot of nuances and trivia, perhaps with inefficient substitutions ( `cmd ...` ), try to write in POSIX Make. For me personally, with many years of experience with GNU / BSD Make, this option is the most time consuming (it's easier to write in several dialects).
- Write in one of the Make dialects, forcing the user to install third-party software.
Make technical problems
But everything is much worse because any Make does not say that it (well) copes with the tasks assigned to it.
- mtime , Make mtime, . , , Make . mtime ! mtime , , ! mtime — , . FUSE mtime . mmap mtime… -, msync ( POSIX ). NFS? , Make : ( ), , FUSE/NFS/mmap/VCS.
- . ? Make . :
tgt-zstd: zstd -d < tgt-zstd.zst > tgt tgt-fetch: fetch -o tgt-fetch SOME://URL
, , Make , , , , Make, .
:
tgt-zstd: zstd -d < tgt-zstd.zst > tgt-zstd.tmp fsync tgt-zstd.tmp mv tgt-zstd.tmp tgt-zstd
tmp/fsync/mv ? , Make-, tgt.tmp. - . ( ) Makefile, Make ? . - $(CFLAGS)? .
Makefile! . Makefile , , . , , - , .
Makefile :
$ cat Makefile include tgt1.mk include tgt2.mk ...
. ? !
- , . Recursive Make Considered Harmful , Makefile-, Makefile- - , , Make , . Makefile — . ? , Makefile.
? , . FreeBSD , , , , .
- . , #include «tgt.h», .c tgt.h, .c - sed .
tgt.o: tgt.c `sed s/.../ tgt.c`
. .mk Makefile include. ? Make, : .mk , , Makefile- include-.
- Makefile- shell, , - , \\$, , .sh , Make. Make /, shell shell, . ?
Let's honestly admit: how often and how much did you have to do make clean or rebuild without parallelization, because something was not compiled or not rebuilt contrary to expectations? In the general case, of course, this is not due to ideally correct, correctly and fully written Makefiles, which speaks of the complexity of their competent and efficient writing. The tool should help.
Redo requirements
To move on to the description of redo , I will first tell you what it is as an implementation and what the "user" (the developer who describes the goals and dependencies between them) will have to learn.
- redo, , - . redo . POSIX shell . Python . : , , .
- redo : POSIX shell, GNU bash, Python, Haskell, Go, C++, Inferno Shell. .
- C , SHA256, 27KB. POSIX shell 100 . , POSIX shell redo tarball- .
- Make-, ( ).
redo
The target build rules are a regular POSIX shell script in target_name.do . Let me remind you for the last time that it can be any other language (if you add a shebang) or just an executable binary file, but by default it is a POSIX shell. The script is run with set -e and three arguments:
- $1 —
$2 — ( )
$3 —
redo . stdout $3 . ? - , - stdout. redo:
$ cat tgt-zstd.do zstd -d < $1.zst $ cat tgt-fetch.do fetch -o $3 SOME://URL
, fetch stdout. stdout , $3. , fsync . ! , fsync — .
, (make) clean, , . redo , . , all .
default
. POSIX Make .c:
.c: $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $<
redo default.do , default.---.do. Make :
$ cat default.c.do $CC $CLFAGS $LDFLAGS -o $3 $1
— $2 , $1 «» redo . default- :
a.b.c.do -> $2=a.b.c default.do -> $2=a.b.c default.c.do -> $2=a.b default.b.c.do -> $2=a
, , . cd dir; redo tgt redo dir/tgt. .do . , .
-.do , default.do . , .do ../a/b/xtarget.y :
./../a/b/xtarget.y.do ./../a/b/default.y.do ./../a/b/default.do ./../a/default.y.do ./../a/default.do ./../default.y.do ./../default.do
2/3 redo .
redo-ifchange :
$ cat hello-world.do redo-ifchange hello-world.o ../config . ../config $CC $CFLAGS -o $3 hello-world.o $ cat hello-world.o.do redo-ifchange hw.c hw.h ../config . ../config $CC $CFLAGS -c -o $3 hw.c $ cat ../config CC=cc CFLAGS=-g $ cat ../all.do # , , <em>redo</em>, # hw/hello-world redo-ifchange hw/hello-world # $ cat ../clean.do redo hw/clean $ cat clean.do rm -f *.o hello-world
redo : state. . redo-ifchange , - , - , , , , . .do . , config hello-world .
state? . - TSV-like -.do.state, - , .redo , - SQLite3 .redo .
stderr - , - state, « - ».
state? redo : , FUSE/mmap/NFS/VCS, . ctime, inode number, — , .
state lock- Make — . ( ) state lock- . .
, redo-ifchange - , . — . redo-ifchange , :
redo-ifchange $2.c gcc -o $3 -c $2.c -MMD -MF $2.deps read deps < $2.deps redo-ifchange ${deps#*:}
, include-:
$ cat default.o.do deps=`sed -n 's/^#include "\(.*\)"$/\1/p' < $2.c` redo-ifchange ../config $deps [...]
*.c?
for f in *.c ; do echo ${f%.c}.o ; done | xargs redo-ifchange
.do (....do.do ) . .do $CC $CFLAGS..., « »:
$ cat tgt.do redo-ifchange $1.c cc ./cc $3 $1.c $ cat cc.do redo-ifchange ../config . ../config cat > $3 <<EOF #!/bin/sh -e $CC $CFLAGS $LDFLAGS -o \$1 \$@ $LDLIBS EOF chmod +x $3
compile_flags.txt Clang LSP ?
$ cat compile_flags.txt.do redo-ifchange ../config . ../config echo "$PCSC_CFLAGS $TASN1_CFLAGS $CRYPTO_CFLAGS $WHATEVER_FLAGS $CFLAGS" | tr " " "\n" | sed "/^$/d" | sort | uniq
$PCSC_CFLAGS, $TASN1_CFLAGS? , pkg-config, autotools!
$ cat config.do cat <<EOF [...] PKG_CONFIG="${PKG_CONFIG:-pkgconf}" PCSC_CFLAGS="${PCSC_CFLAGS:-`$PKG_CONFIG --cflags libpcsclite`}" PCSC_LDFLAGS="${PCSC_LDFLAGS:-`$PKG_CONFIG --libs-only-L libpcsclite`}" PCSC_LDLIBS="${PCSC_LDLIBS:-`$PKG_CONFIG --libs-only-l libpcsclite`}" TASN1_CFLAGS="${TASN1_CFLAGS:-`$PKG_CONFIG --cflags libtasn1`}" TASN1_LDFLAGS="${TASN1_LDFLAGS:-`$PKG_CONFIG --libs-only-L libtasn1`}" TASN1_LDLIBS="${TASN1_LDLIBS:-`$PKG_CONFIG --libs-only-l libtasn1`}" [...] EOF
- .do , Makefile:
foo: bar baz hello world .c: $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $<
:
$ cat default.do case $1 in foo) redo-ifchange bar baz hello world ;; *.c) $CC $CFLAGS $LDFLAGS -o $3 $1 ;; esac
, default.do . .o ? special.o.do, fallback default.o.do default.do .
redo , , « , !?» ( default ). , , , , . suckless ( , CMake, GCC, pure-C redo — ).
- - .
- (*BSD vs GNU) — POSIX shell , (Python, C, shell) redo .
- / Makefile-.
- .
- ( ) , , .
- — , , l **.do.
/?
- Make , .
- It took me more than one month to unlearn the reflex to do redo clean , as it’s already a habit after Make that something will not (re) gather.
I recommend the apenwarr / redo implementation documentation, with tons of examples and explanations.
Sergey Matveev , cypherpunk , Python / Go / C-developer, chief specialist of FSUE STC Atlas.