I went, and I put together 3 quality of life snippets I use all the time and put it in a single makext.mk file that can be included in other Makefiles and wrote a basic readme for it.
This is not meant to be a replacement for other task runners, but I do think it can be useful to some of you.
https://github.com/mitjafelicijan/makext
Check it out and see if it makes sense to you.
Thanks for any feedback or comments.
Cheers
Whenever I see comments about people trying to reinvent make, my conclusion is that make "as a tool" is inevitably far superior, but people are mostly familiar with a small subset of its functionality, which makes it appear clunky. That, and the syntax is very flexible, which means it can easily be abused to write ugly/convoluted makefiles.
If I had one thing I'd want to improve about make, it wouldn't be the tool itself, but its documentation, and a set of reasonable defaults/templates, like you have here.
The documentation is amazing in one sense, in that it's very comprehensive, but very clunky and hard to master in another, in that it's organised in a very high-level manner that doesn't allow it to be used as a quick reference / point of truth, if you don't know what you're looking for. I found myself making anki notes for it, as it's not organised in the kind of way that would simply allow me to refer to the docs for your usecase and get things done. You really either know some functionality exists and can look it up or you don't, and unless you're someone who uses make in expert mode on a daily basis, it's hard to know all the specialised components and be able to combine them all together seamlessly. Hence my anki notes, hahah.
But make really is an amazing tool. I wish people focused on improving make workflows instead of reinventing the wheel all the time with other tools (which are then forced on the user).
If you're looking for a task runner, not a build tool, then make is not "far superior" in any sense and there are much better alternatives, the most prominent probably being "just". Even something as basic as accepting parameters to a task (make fetch <arg1> <arg2>) is awkward because make doesn't understand this syntax and interprets the arguments as targets. The only way to make it work is either through named arguments or empty target hacks and in both cases you need to write the validation logic yourself. In just it's simply:
fetch PACKAGE VER:
@echo fetching {{PACKAGE}} at version {{VER}}
$ just fetch foo
error: Recipe `fetch` got 1 argument but takes 2
usage:
just fetch PACKAGE VER
$ just fetch foo 1.2.3
fetching foo at version 1.2.3
AFAIK, Just doesn't have any way to mark "this task is complete, so don't re-run it". It also doesn't have file-based dependendicies, which are pretty common for any kind of programming.
Why would I want to lose all of Make's flexibility and power in exchange for slightly prettier UI syntax?
----
fetch PACKAGE VER:
@echo fetching {{PACKAGE}} at version {{VER}}
----
$ just fetch foo 1.2.3
----
fetch:
$(if $(PACKAGE),,$(error PACKAGE was not set))
$(if $(VERSION),,$(error VERSION was not set))
@echo fetching $(PACKAGE) at $(VERSION)
----
$ make fetch PACKAGE=foo VERSION=1.2.3
As a personal preference, I actually do tend to avoid Make as a task runner because a quick shell script is almost as easy to write and far more flexible. I am pretty familiar with Make but have still painted myself into a corner with it more times than I'd like to admit.
But also it's never something I have to worry about. Pretty much every syntax-aware text editor knows how to handle Makefiles, and knows that recipe steps start with a tab (unless you change Make's default). So for me, that falls in the "ugly, but not really a usability issue" bucket. And Make's power/expressiveness is worth accepting a few "I'd have done it differently" syntax things.
a good taskrunner makes it easy to run a task while still exposing the tool's underlying flexibility. a good taskrunner lets me invoke the entire test suite with a short command, but also allows me to add custom options and arguments to, say, run a specific test case in an alternate environment.
`make` fails to expose the tools' underlying flexibility. sure, you can write a .PHONY target to run the full test suite, but `make` can't handle passing options or arguments (besides cumbersome Makefile variables).
a makefile tends to obscure the underlying tool, enshrining its launch arguments. anyone who has tried to cross-compile a makefile codebase authored by someone who didn't consider cross-compilation understands what i mean (you'll end up re-writing the Makefile 90% of the time).
I agree with _everything_ else you've said though.
I don't like make for various reasons, but portability is not one of them.
i admit my tirade is a little less applicable to C projects; many C projects are already using `make` for legitimate reasons, and at that point are already making some POSIX-y assumptions. so if that project's taskrunner needs are light, it really might make sense to just cram them in the Makefile and mark them .PHONY, despite make's shortcomings as a taskrunner.
my portability comments were really written with windows in mind.
And no tool is perfect. I think it's all about balance. Example: I love Zig build tool, but I think it would scare some of the people who would just want to add something to the build pipeline and have no idea about Zig. Projects are rarely just the code they have. They have scripts and require installing additonal software.
But yeah, I agree with your points here.
What it might look like in a pyproject.toml:
[tool.taskipy.tasks]
fmt = "nox -s fmt"
lock = "nox -s lock"
install = "if [ $VIRTUAL_ENV ]; then pip install -r local-requirements.txt; else printf '%s\n' 'Please activate a venv first'; return 1; fi"
test = "nox -s test test_without_toml typecheck -p 3.12"
docs = "nox -s render_readme render_api_docs"
I stumbled over this statement in your README:
> It is recommended to use .PHONY for targets that are not actual files. In the example below I am not doing that though, but it is wise to follow that rule.
Please also give the example as you think it would be correct. Or write the understandable reason why you don't do it here. I think many people don't read the text between the code snippets but only scan the code examples. It would be helpful to find the ‘best practices’ right there.
It's not, don't apologize, reusing battle tested tech is the way.
I have a quite close "template Makefile", though I prefer to store it as a cookiecutter (or more recently "copier") template for easier per-project bootstrap.
You could add a timestamped log function. I use that in mine so that targets have timestamped messages pre/post.
It's a little more user-friendly, but not enough to justify how much power you lose.
I spent the last 20 minutes reading Just's documentation, and it seems like the only real wins are nicer argument parsing and && dependencies, and the --list argument. And in exchange for that, you lose file-based dependendicies and Make's powerful templating engine. Did I miss some other features that make the trade worth it?
But often tasks are a step removed from that. Targets like "all" and "clean" are good examples of useful tasks to have, even if they're unrelated to files.
A "task runner" isn't intended to replace all of a Makefile's functionality. It takes a common use case of Makefiles, and improves the user experience for this.
I think it's fair to say that it's not a big step up; but it's a definite improvement for what it does aim to do.
Some of the issues could be avoided by requiring a -- and using the .SILENT modifier, but some are extremely difficult: e.g. it word-splits strings and you can't just pass "some string" as a single argument, everything will be separated by spaces.
In case the underlying program is a script-language wrapper that you also control, it is possible to hack around and let Make pass through its own $$PPID so that the underlying script could read the nul-terminated /proc/<Make's pid>/cmdline and pass it on to the actual program.
This, of course, only works where procfs is a thing, so e.g. on macOS (if it's among your target platforms) you'd have to reinvent the wheel and learn a whole lot of Darwin sysctl dark magic to read arbitrary process' arguments, at which point you'd ask yourself if this is even something a sane person would ever do...
A lot of people recommend just; here's their sample file:
alias b := build
host := `uname -a`
build:
cc *.c -o main
test-all: build
./test --all
test TEST: build
./test --test {{TEST}}
#!/usr/bin/env sh
set -eu
b="build"
host=`uname -a`
_cmd_build () { # build: Build main program
cc *.c -o main
}
_cmd_test_all () { # test_all: Run all tests
_cmd_build
./test --all
}
_cmd_test () { # test TEST: Run a test TEST
_cmd_build
./test --test "$1"
}
_cmd_help () {
echo "Available targets:"
grep -E "^_cmd_[^[:space:]]+ \(\) {.+" "$0" | sed -E 's/.*#/ /'
}
[ $# -gt 0 ] || _cmd_help
set -x
"_cmd_$1" "$@"
$ chmod 0755 run.sh
$ ./run.sh
Available targets:
build: Build main program
test_all: Run all tests
test TEST: Run a test TEST
./foo.sh: line 23: $1: unbound variable
$ ./run.sh build
$ ./run.sh test_all
$ ./run.sh test foo.t
- It handles all that boilerplate for you. The only stuff in the justfile is code you want to execute, not the code to figure out which code to execute.
- Out-of-the-box tab completion. "What did I name that recipe? `just t<tab>` Oh, `test-all`, that's right!"
I like not reinventing those wheels. Let someone else manage that hassle for me.
[Apologies for contributing to the slightly off topic discussion. makext sounds great for people still using make for this and I totally would have used it in the past.]
For smaller projects I like make and people who work with me and know me always first check Makefile to see what is there and what the project is about. I use it as a launchpad for installing dependencies and things like that.
And it's all about what is appropriate for that project. Something a shell script is better. Sometimes soemthing else. So I completely understand what you are saying.
Both are extensions to classic make (makefiles without advanced features works with both implementations), but, to my eye, pmake extensions is much more simpler and nicer to use, than GNU make extensions.
Simple ".if/.else/.endif" and true ".for" loop, like in sh is much more comprehensible than GNU make functional extensions to me.
Is it only me?
And BSD systems have extensive library for pmake already, which allows to build programs, libraries (both shared and static) and more in declarative way, without any custom rules. It is like these extensions, but battle-tested for 20+ years. With nice license, too.
Here are a few settings that make GNU make a bit faster, and enable multi-line, strict-mode bash scripts as recipes (make recipes are normally sequences of single-line sh invocations)--and enable quietude & tracing: https://github.com/barries/polling_state_machine_cpp/blob/ma...
I also believe I've seen someone setup tools that can draw the dependency graph for you, so you can view (not edit) it airflow style.