I’ve been running Unix systems for 25 years. All of the things you mentioned were already solved adequately in the Linux 0.9 days, and the BSD’s still use old fashioned shell based inits, and work fine in my experience.
In fact, the only time I’ve had issues with any of this stuff has been in the last ten years, and only on systemd-based distros.
The genius of init systems is that they factor all this stuff out, and just handle starting and stopping daemons.
really?
Let's say (not entirely hypothetically) that you have a daemon. This daemon has a config file, which is large and complex. This daemon's /etc/init.d file builds command-line dynamically, then passes it to start-stop-daemon as usual.
Puzzle level 1)
You made a typo in init file, so that daemon does not start. The "start" command succeeds, but there is no output anywhere, and process does not start. How do you debug it in the sysvinit world?
Puzzle level 2)
Turns out this daemon used LC_ALL (PATH? HOME?) when parsing config, so it works fine when started or restarted from command line, but fails when started as an part of startup process. How do you debug this?
Puzzle level 3)
One day, you happened to be in the server room, watching server boot (you normally reboot it remotely). You have noticed a red message scroll up the screen. This looks important! Too bad it already scrolled off the screen and out of the scrollback buffer too. How do you see it?
The problem of all the init systems is they could be great, but their development just stopped at some point -- and instead of fixing the real annoyances in a general way, they were just saying "this is all fine". Until systemd came and ate their lunch.
> You made a typo in init file, so that daemon does not start. The "start" command succeeds, but there is no output anywhere, and process does not start. How do you debug it in the sysvinit world?
"oh my process failed; let's add `-x` to the invocation."
> You made a typo in init file, so that daemon does not start. The "start" command succeeds, but there is no output anywhere, and process does not start. How do you debug it in the sysvinit world?
as above.
> One day, you happened to be in the server room, watching server boot (you normally reboot it remotely). You have noticed a red message scroll up the screen. This looks important! Too bad it already scrolled off the screen and out of the scrollback buffer too. How do you see it?
`dmesg` or /var/log/messages (or /var/log/dmesg)
you mean, "add -x to the invocation, get back a few screenful of text, wade through that looking for "export" lines and copy them to your terminal, find "start-stop-daemon" line, copy the part of the line and remove irrelevant parameters, and paste into your terminal as well?
I would not call this "easy" or "solved". Possible, yes. Pleasant, no.
Puzzle level 2) inherited env -- I think you made copy-paste error? You just copied the same paragraph again.
Puzzle level 3)
Surprise! Neither dmesg or /var/log/messages or /var/log/dmesg usually contain startup problems! Sometimes the nice daemon might log stuff there, but it is no way universal or required.
The sysvinit recommended way is to call things like "log_failure_msg" and "log_warning_msg", and those only print to stdout, which is the console. That's it. Gentoo's OpenRC does claim to have boot console logger, but it was broken every time I tried to use it.
Are you running any servers? If so, you might want to connect to the monitor, reboot and see if any service has error messages. Or you can use console scrollback after the server is booted, but you might need to recompile your kernel to increase is size, it is pretty small. Or just install systemd, it has full boot logging fully integrated with primary logging. And even a special command which lists all daemons which failed to start.
I mean, yes? Granted I'm not a developer but printing out exactly what's happening line by line with all variables expanded and all logical conditions printed is awesome, it's something we try to reverse engineer into programs with tools like bpftrace and strace.
Conversely an opaque C application which fails has much less tooling in the way of debugging failures. Especially if your package maintainers have stripped symbols.
> Surprise! Neither dmesg or /var/log/messages or /var/log/dmesg usually contain startup problems! Sometimes the nice daemon might log stuff there, but it is no way universal or required.
Actually, that's the opposite. A "bad" daemon might not print anything to stdout when invoked (which goes to dmesg) but regardless, this whole comment just reeks of someone who hasn't actually dealt with systems before 2014.
> Are you running any servers? If so, you might want to connect to the monitor, reboot and see if any service has error messages. Or you can use console scrollback after the server is booted, but you might need to recompile your kernel to increase is size, it is pretty small. Or just install systemd, it has full boot logging fully integrated with primary logging. And even a special command which lists all daemons which failed to start.
Console scrollback is "nice to have", personally I'm very thankful that it does not use memory keeping things there because there is the odd regression when you configure something (like when you tell IPtables to log deny's: that also gets printed to console.)
re----gardless. I think we're talking passed each other. I had no real issue as a sysadmin with a fully fledged bash, because I can debug it. I get that there's issues with that and most people do not like it- I'm not advocating for it, just saying that what you presume to be "impossible" was somehow solved for decades, and quite competently. It may not be pleasant to you but the alternative is not pleasant for me.
The larger issue I have is that "the alternative" is embedded and entangled itself to everything- there can never be anything better produced.
With old init's, it was a small process with well defined functionality so replacing it was easy. (as evidenced by the many init's among unices.).
Anyway, I'm getting ahead of myself.
If you want to see what systemd could have been, I invite you to look at Solaris' SMF. Please ignore the XML-ness of it, it's "very old" in comparison to systemd. :)
