As much as I would like to support efforts like this, I feel like it is ultimately doomed to suffer from usability limitations because make does not have a static DAG. Rather, the DAG evolves dynamically as make evaluates targets. There are pesky problems like $(call) and $(eval) where you can dynamically inject make expressions into a partially evaluated DAG. And targets can generate files which are later loaded via include directives.
Dumping the make database in a machine readable format (for visualization or otherwise) would be incredibly valuable for debugging and could improve understanding. But since many make files have N>>1 "snapshots" of the internal DAG during their execution, there is a really thorny problem around when and which of these "snapshots" to use and how to stitch them together. Many make files aren't "static" enough to support exporting a single, complete, and valuable usable snapshot of their internal DAG.
If debugging is the target goal, I think a potentially better approach would be to define a function that takes an output filename and optional list of targets and writes out the point-in-time build graph when that function is called. This way makefile authors could insert probes in the make file (including at the bottom of a make file so the function is called on file load) and capture the state(s) of the build graph exactly when they need to. There could also be a specially named variable holding the names of targets that when pre- or post-evaluated would trigger the dumping of the database.
Best of luck to the person doing this work. Debugging make is notoriously hard and any progress in this area will be much welcomed by its many users.
If you can get away with it use -rR to skip builtins, it makes the output of -d much nicer.
I usually end up running make with -rRd or -rRp when I want to inspect things. -d tells you why things happen and -p shows you what rules and variables are defined.
Not only can you see what was rebuilt and why, you can see all the input into the rules at all the stages of the build, conveniently hyperlinked into the master rule files provided in the SDK.
i remember reading about some wild system on here that exposes the entire source tree via fuse so that it can enforce rules about what changes and what doesn't for each step. wish i could remember the name...
edit: ah, yes. tup. https://gittup.org/tup/
edit 2: see tup graph: https://gittup.org/tup/ex_dependencies.html
You can get a .drv file using 'nix-instantiate' (or 'nix repl'), e.g.
$ nix-store --query --tree "$(nix-instantiate -E '(import <nixpkgs> {}).bash')"
/nix/store/378a0jpkxiwzyk6jj6w2y2qdpn8790nc-bash-4.4-p23.drv
+---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv
| +---/nix/store/71d9qf26s1nzrp34cqw1hh0yrx0h1kbx-cpio.drv
| +---/nix/store/7jl6nhfsc0mchccg5w40rfm4zfshpsag-bzip2.drv
| +---/nix/store/bci5x35014bbpc87x7ds2n2ffh5syns1-sh.drv
| +---/nix/store/fnn1nmssk7fsfn83ik66cxddqxzvnwx6-bootstrap-tools.cpio.bz2.drv
| +---/nix/store/p1bhqmlpf6vr9bc5mb99w29mvllcqmsm-mkdir.drv
| +---/nix/store/spcnl3whhm392kir1snaar4wlcawr3yq-unpack-bootstrap-tools.sh
+---/nix/store/04npl9bjcjkpqgwx1s94azbimfgr531v-bash44-012.drv
| +---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv [...]
| +---/nix/store/z6aidzdk3yl2k92g8qj8xgzr8glp0vjr-bootstrap-stage0-stdenv-darwin.drv
| | +---/nix/store/1i5y55x4b4m9qkx5dqbmr1r6bvrqbanw-multiple-outputs.sh
| | +---/nix/store/bnj8d7mvbkg3vdb07yz74yhl3g107qq5-patch-shebangs.sh
| | +---/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
...
https://man.archlinux.org/man/community/nix/nix-store.1.en#S...
ninja -d explain
?I've been using this in my build system to make the explain output slightly more useful: https://github.com/ninja-build/ninja/pull/2067
It's just a quick hack that took me half a day (most of which was remembering how to C++ after 8 years...) but I think it makes "-d explain" much more useful.
Really, the "explain" implementation could do with an overhaul. I'd love ninja to serialise the build graph to disk (or maybe just the subset of the graph that actually needed rebuilding). This could be augmented with the reason why each rule was run, and possibly even with the command output for each rule. Then the existing ninja tools like "graph", "browse", and "query" could optionally use this graph instead of the "build.ninja" file, so that you can inspect what happened the last time you ran "ninja".
Realistically I'll never get around to implementing this, and my pull request above gets me close enough.
I use it like this from inside the Makefile:
make-db.png:
make -f $(MAKEFILE) -Bnd | sed -e 's,$(MAKEDIR)/,,g' | make2graph | dot -Tpng -o $@
bazel query ${some_expression} -output graph | dot -Tpdf > out.pdf
https://techpubs.jurassic.nl/manuals/0650/developer/Debugger...
https://www.cmcrossroads.com/article/tracing-rule-execution-...
Also I feel like on most nontrivial projects the graph would be gigantic & to get any usability you'd need something interactive where you can collapse/expand the nodes you're (not) interested in. Don't we have a bunch of JS libraries these days that provide this?
ISTR there was a thread about it here a few months ago.
> The Android build system works by including all recipes to be built (programs / libraries / etc) using the GNU Make include directive, so that you end up with one giant Makefile that holds all rules for building the platform. Possibly to avoid the problems laid out in the paper Recursive make considered harmful.
The old build system (ndk-build) works like this.
Modern NDK applications are supposed to use CMake and Android's own build system is based on Soong.
Soong build files are easy to parse and you can create an Unified Dependency Graph from them. That's what I did in BGraph [0] for security applications.
That said, isn’t there some software to visualise dot files in a more sophisticated way than just generating a picture? It would be handy to be able to collapse/expand sub graphs in a GUI, for example.
There's a lot more to Graphviz than just dot, as it comes with some really good filtering tools that people often seem to miss. You can achieve an awful lot with just bcomps/ccomps. And, when you're running queries gc is useful to see that you're heading down the right path.
(One of the advantages of DOT is that it's a pretty common de facto standard for describing graphs, so you can import it into most graph manipulation libraries and mangle it further using that.)
I’ll try the fork.