The author of this piece makes the classic mistake of equating the init system as the process manager and process supervisor. These are, in fact, all separate stages. The init system runs as PID 1 and strictly speaking, the sole responsibility is to daemonize, reap its children, set the session and process group IDs, and optionally exec the process manager. The process manager then defines a basic framework for stopping, starting, restarting and checking status for services, at a minimum. The process supervisor then applies resource limits (or even has those as separate tools, like perp does with its runtools), process monitoring (whether through ptrace(2), cgroups, PID files, jails or whatnot), autorestart, inotify(7)/kqueue handlers, system load diagnostics and so forth. The shutdown stage is another separate part, often handled either in the initd or the process manager. Often, it just hooks to the argv[0] of standard tools like halt, reboot, poweroff, shutdown to execute killall routines, detach mount points, etc.
To stuff everything in the init system, I'd argue, is bad design. One must delegate, whether to auxiliary daemons, shell scripts, configuration syntax (in turn read and processed by daemons) or what have you.
sysvinit is certainly inadequate. The inittab is cryptic and clunky, and runlevels are a needlessly restrictive concept to express what is essentially a named service group that can be isolated/overlayed.
Of course, to start services on socket connections, you either use (x)inetd, or you reimplement a subset or (partial or otherwise) superset of it. There's no way around this, it's choosing to handle more on your own rather than delegate. In systemd's case, they do this to support socket families like AF_NETLINK.
As for systemd being documented, I'd say it's quote mediocre. The manpages proved to be inconsistent and incomplete, and for anyone but an end user or a minimally invested sysadmin, of little use whatsoever. Quantity is nice, but the quality department is lacking.
sysvinit's baroque and arduous shell scripts are not the fault of using shell scripts as a service medium, but have to deal with sysvinit's aforementioned cruft (inittab and runlevels) and the historical lack of any standard modules. BSD init has the latter in the form of /etc/rc.subr, which implements essential functions like rc_cmd and wait_for_pids. Exact functions vary from BSD to BSD, but more often than not, BSD init services are even shorter than systemd services: averaging 3-4 lines of code.
A unified logging sink is nothing novel, it's just that systemd is the first of its kind that gained momentum, but with its own unique set of issues. syslogd and kmsg were still passable, and the former also seamlessly integrated itself with databases.
Once again, changing the execution environment is a separate stage and has multiple ways of being done. Init-agnostic tools that wrap around syscalls are probably my favorite, but YMMV.
As for containers, it's about time Linux caught up to Solaris and FreeBSD.
The process manager gets killed. How do you recover?
If you have respawn logic for it in PID 1, how do you log information about a failure to respawn the process manager?
Perhaps you build in some basic logic for logging. Where do you store the data? What if the user level syslog the user wants you to feed data to can't be brought up yet, because it depends on a file system that is not yet mounted?
There may very well be alternatives to the systemd design, but I've yet to see any that are remotely convincing, in that most of them fail to recognise substantial aspects of why systemd was designed the way it is, and just tear out stuff without proper consideration of the implications.
Most proposed alternative stacks to systemd falls down on the very first question above.
I agree with you that it doesn't seem like a great idea to stuff everything in the init system, but I don't agree that "one must delegate" unless the delegation reduces complexity, and I've not seen any convincing demonstrations that it does.
I'd love it if someone came up with something that provided the capabilities and guarantees that systemd does with indepenent, less coupled component, though.
But there's no way I'm giving up on the capabilities systemd are providing again.
What happens if the process manager crashes if you're running systemd: the process manager is in PID1 (or, equivalently, in a tightly coupled process that PID1 depends on - because the whole point of your post was that you can never get to a state where PID1 is working but logging isn't working), so your system crashes, every time. How is that better? And if that's really what you want, it's easy to configure a decoupled init system to do that.
Hey, some people like their logs to be sent as email. Maybe we should move sendmail into PID1 as well.
> The author of this piece makes the classic mistake
> of equating the init system as the process manager
> and process supervisor.
In the interest of full disclosure I must admit I was on duty when AT&T and Sun were creating the unholy love child of System V and BSD, I'm sorry.
The architecture, as bespoke by AT&T system engineers, was that process 1 was a pseudo process which configured the necessary services and devices which were appropriate for an administrator defined level of operation. Aka a 'run level.' I think they would have liked the systemd proposal, but they would no doubt take it completely out of the process space. I am sure they would have wanted it to be some sort of named stream into the inner consciousness of the kernel which could configure the events system so that the desired running configuration was made manifest. They always hated the BSD notion that init was just the first 'shell process' which happened to kick off various processes that made for a multi-user experience.
Originally users were just like init, in that you logged in and everything you did was a subprocess of your original login shell. It was a very elegant system, root's primal shell spawned getty, and getty would spawn a shell for a user when they logged in, everything from that point on would be owned by the user just like everything that happened before was owned by root. The user's login shell logged out and everything they had done got taken down and resources reclaimed. When the root shell (init) logged out all resources got reclaimed and the system halted.
But Linux, like SunOS before it, serves two masters. The server which has some pretty well defined semantics and the "desktop user" which has been influenced a whole bunch by microcomputer operating systems like Windows.
I wasn't the owner of the init requirements document, I think Livsey was, the important thing was that it was written in the context of a bigger systems picture, and frankly systemd doesn't have that same context. I think that is what comes across as confusion.
Not to mention, most of the systemd hate seems to be spread by only two main sources now, and both cite each other as sources (ironic a little).[1]
[1] http://www.jupiterbroadcasting.com/66417/systemd-haters-bust...
systemd was really designed with servers in mind, and really does bring a lot to the table for server admins.
The author is not making any mistake at all, or no more so than you are.
I'm sure you both value engineering principles like separation of concerns and a single source of truth.
The author believes that by removing the redundancy between initd / xinetd / supervisord / syslog the system is improved.
You disagree, and believe that these are separate concerns.
That's fine, you have different values / judgements in this matter. But saying he's `mistaken` for not agreeing with you is childish.
The dead simple rc.conf file seems so much nicer than the stuff I was dealing with in the entire world of Linux-based systems, like going back to the way Arch used to be when I really liked it.
We always need strong alternatives, even if they face the risk of being taken as simply a political statement, the effects of the statement will be seen in the decision-making down the road.
There are some real issues being pointed out (particularly regarding monolithic design) but no-one has attempted to actually fix that in any way (in code, that is).
While it is unlikely that I will end up using uselessd (unless it "wins" in some way, e.g. in embedded space with uclibc and musl), I very much welcome the effort to bring out alternatives that address the same problems as systemd, yet trying to fix some of the issues there are.
You've slayed the straw man... ;-)
systemd doesn't put everything in pid 1. It defines some mechanisms to orchestrate the whole thing that include pid 1.
I prefer the BSD-style philosophy, nice, simple rc.conf, used to run Arch till it got infected with this garbage too. It slowly progressed away from it's BSD-style roots. So recently, I just gave up and moved to FreeBSD. Not a single regret so far.
My only issue with journald's binary log files is that they're of homebrewn custom format, that's not accessible by any standard means. Plain text files aren't directly readable by humans, too, but we have cat, less or alike tools to pass such data to terminal (sometimes an iconv is required, say, if log entries contain filenames that has characters outside of ASCII range), and those tools are available on every modern OS out there.
Personally, I think, a compromise that'd satisfy me (YMMV) would be either an industry-standard log format (like, maybe, sqlite - it's fairly simple, universal and omnipresent nowadays) or, even better, storing data in text files, but having accompanying binary index and metadata ones that store non-human-readable stuff (like hash chain - bet, no sane human would ever check cryptographic log integrity by hand) and provide additional information for faster machine access.
But the heck journald is a tightly-coupled part of into systemd instead of being a separate project is beyond me. I can't reject that systemd has some good things about it too, but it's too terribly monolithic and unhackable compared to mostly-scripted init systems. And such negative points easily outweigh the positive ones.
I agree. There is a strong need for common interface(s), and that's a strong part of the motivation behind Fluentd/Kafka/etc. www.fluentd.org/blog/unified-logging-layer
And a journal that has another journal inside it would be somewhat silly. A simple write log can be done better.
I don't think fsync on every log commit is a good idea. This would be more a DOS attack.
That said I'm somewhat troubled by the cavalier attitude to log data safety in journald too.
This seems to come up frequently. I'm curious: how alternatively would a local process communicate with the Init daemon?
Why is the init daemon what a local process should communicate with?
A daemon that managaes other daemons doesn't need to be PID 1, even to reap zombies.
Secondly, daemons shouldn't care what process is managing them, a principled approach to communication between the daemon manager and a daemon would probably include handing off sockets/ports/fds, but probably not much else.
The answer to that is easy: It's still signals, like it has been for decades, and systemd doesn't change that any more than anything else does. systemd defines some more signals, but that's about it.
Notice that you said init daemon. The errors here are in thinking that (a) dbus is the communications system, even in systemd, for the part of the system that does overall system state management and the stuff that the kernel requires of process #1; and (b) the part of the system that supervises daemons is in all packages run in process #1, the "init daemon".
systemd, the package, uses dbus in a number of places between a number of components, most notably logind, hostnamed, timedated, machined, and localed. But don't get that confused with systemd the program that runs as process #1, which is as constrained by the kernel (and others) into using the same signals as always, just as any other system manager is. There's an AF_LOCAL socket (/run/systemd/private) for systemctl to talk to PID #1 using a private undocumented protocol, and a public documented D-Bus API for units, but those are for the service management part of systemd the program.
The system manager and service manager in the nosh package are in the same boat. The system manager's API is that same set of signals again (augmented with some of the extra systemd-defined signals that fit the model). There's no other API because the system manager is not what the world talks to about service management. The control/status API for individual services is the filesystem, the service manager (that is not in PID 1) presenting the same suite of pipes and files as daemontools-encore. And there's an AF_LOCAL socket for system-control (and indeed anything else, such as service-dt-scanner a.k.a. svscan) to talk to the service manager for loading and unloading service bundles in the first place, and plumbing them together.
It all depends on the use cases.
For me, FreeBSD is a no go given my desktop usage requirements.
I'm just glad for Slackware at this point.
I hope systemd will live or die on its merits; I fear that it will take over via politicking.
Mostly I feel that it got an undue jumpstart thanks to RedHat trying too hard to be bleeding edge, and Upstart/OpenRC not having as high-profile a backer.
Canonical's insistence on control and ownership ended up torpedoing its project. Which is really quite sad.
I wised up and moved over to using the Debian distribution in this case. Less moving parts trying to make things work the way they thought they should.
Also, it's being rapidly iterated and there isn't a PPA for Ubuntu, which sucks.
So if I follow you, it would be easier to grok the whole system if all the code was in different places?
This is more difficult if you have a lot of abstractions and binaries lying around. You need to start reading (often nonexistent) documentation and abstract C-code...
It likely does not matter for 95% of users and I've rarely had to do something like this but you are losing some control as developer/sysadmin. For some it's important as their productivity and job depends on solving such issues fast, others will never have this problem...
I've never had a problem with systemd through. But if you have it's difficult to fix it on your own.
Yes. Separated in documented modules that are self-contained, with well defined behavior. When your logger breaks, you fix your logger. Not your init process. When your devices are not discovered you debug udev not muddle through a code riddled with synching with ntp and so on.
It's very "UNIX" to implement things as communicating processes rather than RPC or procedure-call-within-monolith.
However, in my mind it has made several awesome things possible. My boot time got dramatically shorter when I adopted it thanks to parallelization. Besides, daemons have now simple and robust service definitions. Sys V had become a mess!
Lastly, lightweight containers are the real-deal for small development tasks (not for production!). Just one command: systemd-nspawn, and you're ready to go. Docker is currently a bit more complicated to set up.
Arguably, many features, including containers, should be moved out of systemd. Right now, more than a monolithic architecture, I think systemd is rather shipping too many things under the same project umbrella.
Writing daemon startup files was somehting I always dreaded, and never really did well.
Before systemd, if I needed to run services I'd try to use daemontools (for auto-restart, and logging), but then I had two service-starting services running my system. Upstart had some of the features, but was still finicky (and the versions I had available didn't consistently have good service supervision support).
systemd just fix that.
Also, with systemd, for the first time I feel like I'm really using Linux, not just a random *Nix that has adequate drivers.
In a server environment it happens that the various DRAC/BIOS(es) are initialized and the bootloader reached is far longer (several minutes sometimes) than the boot time of sysvinit. So optimizing boot time in the Linux part on a server is probably moot for me.
On the laptop you can suspend/hibernate as others have said if you care about startup time. I have full-disk encryption and need to type in password to boot so few seconds more or less doesn't matter anyway.
So that leaves the desktop, where I might care about boot times (the UEFI/BIOS is actually saner than in servers and reaches the bootloader very fast). It turns out that my desktop boots faster than my router with sysvinit already, so having faster boot times on my desktop would get me nowhere, I still wouldn't be able to use the internet until the router has booted.
So faster boot times ... I didn't need all the pain systemd is causing just for that. Debian has haid makefile/startpar based concurrent boot already, I don't think systemd would improve on that much ...
Meanwhile not using systemd breaks things that used to work on a KDE desktop (USB mounting, VPN config, etc.), so having an app support systemd is a net-negative for me.
The fact that BIOS/DRAC/RAID initialisation is slow on some servers is irrelevant. Linux's init and the firmware initialisation don't run concurrently, therefore if init takes longer the whole boot takes longer. Additionally many servers manufacturers have improved boot times in the last few years (down from 10+ minutes, to 5+ minutes, to less).
Most routers don't take as long to boot as you claim. The entire OS is about 8 MB (uncompressed) and RAM is only 32 MB, and the medium that the OS is stored on is faster than a computer's hard drive. So just looking at IO should tell you your supposition is flawed. In my experience most Linux based routers boot the RJ-45 interface (LAN side) in under 20 seconds unless it is allocating slowly on the WAN interface (e.g. unable to get an IP, etc). If you set a static WAN IP/gateway/etc, then boot times comes down substantially.
Additionally the whole concept that every time your PC turns off you'll also turn off your router at the mains is, uhh, strange. Sure there are power cuts but that isn't the only time you shutdown your PC throughout the year.
The concept that your PC needs to wait for the server is equally flawed. Again, yes, power cuts. But PCs get shut down significantly more often than servers and if we're playing that game then wouldn't a "server" have a UPS anyway?
So overall your argument for why boot times don't matter lacks any kind of substance. It is also purely based on a PC->Server->Router infrastructure where nothing is on a UPS and everything suffers from a power cut (then "races" to all come back up).
In the real world my phone has Linux, our "Tivo" has Linux, our printer has Linux, our car's entertainment system has Linux, etc. So bad Linux boot times will be noticed day to day. It matters to a lot of people and while I don't know if systemd is the solution, I do know that progress is needed relative to the classic UNIX init system (per the article).
You might be interested in firejail [1]. It makes finer-grained use of Linux namespaces, and doesn't depend on systemd (or much of anything, for that matter).
I've come across this sentiment a few times in the last month, but I haven't yet heard an explanation other than "VMs are battle-tested and containers might leak data to each other". Is there something more that I'm missing? Why aren't containers a good idea to use in production?
Secondly, assuming a bug in the kernel, one might assume root in a container can lead to root on the host. Bsd jails have been pretty solid for the last few years afaik - but hardware support for virtualization might still get more of both separation/safety and speed. There have been som bad bugs in (as i recall) the io system in xen, leading to similar issues ... but again the last time i saw anything on that was years ago.
Ymmv - generally docker doesn't have "run untrusted code, safely, as root" as a design-goal (yet, afaik) (not entirely sure about lxc, née vserver -- the underlying technology) -- so don't expect it to do that. Isolation and security (esp. without sacrificing performance) is very hard to get right. Or so a long series of privilege escalation exploits across many different os' seem to indicate.
See Dan Walsh articles here: http://www.projectatomic.io/blog/2014/09/yet-another-reason-...
systemd = init system + a whole lot of other things.
When people complain about systemd,they usually do not complain about what it does or how it does it in the init system part.That part is pretty solid as far as functionality is concerned.
When people complain about systemd,they usually complain about the "whole lot of other things" part.Lots of people have different complains and my biggest one is on udev.
udev is a core component in any modern linux system I see systemd absorbing it as nothing but a political move and a power grab.They could have left udev as an independent project and just create a dependency on it.
The "whole lot of other things part" will,by definition,make any other project that is just an init system seem very much deficient in functionality when compared to systemd.
Its not like if they were stupid and took a bad decision because they didn't know any better. They took the bad decisions conscientiously, for these power, political grabs.
That's not how people want Linux distros to be. They want technological innovation to be the driver.
And being on the same camp: fuck you systemd for successfully adding to that model where political profit is more important than technological innovation.
Let's keep some perspective here. We are literally just talking about an init system. There are many others you can use. systemd is not taking away your freedom in any meaningful sense of the word "freedom". Debates should be about the technical merits of system not baseless accusations that they are making a power grab.
This is absolutely incorrect. We're to the point where certain software packages (GNOME comes to mind) are requiring hard dependencies on it. I was just today reading about some incompatibility that arises if your kernel is set up with no IPv6 support which is explicitly caused by systemd. (To which the response from the systemd folks was something along the lines of "You shouldn't be turning it off anyways")
(Great, now our software takes philosophical positions...)
Sure, you're "free" to use something else, in the same way that you're "free" to patch and recompile every program that touches it to stop touching it. So "free" in the FOSS sense that nobody but developers care about.
Meanwhile, in the real world, populated by end users and sysadmins, the most important people when it comes to a computing environment, the ones that all of this crap is being done for at the end of the day... not so much.
sigh
I'm annoyed that systemd is taking over for political reasons and not purely on its technical merits, and that there is no way this is not going to lead to a monoculture. There will be others, but they will be relegated to the position of marginalized, niche players that nobody outside of /g/ troll threads care about.
I'm annoyed that the rest of the world is going to have to adapt to this software, rather than the other way around.
I'm annoyed that this software is doing 5000 things where one would do.
Systemd does more than just init, it has the features to replace everything from network manager to fstab. It is not just an init system it is an invasion of Linux userspace.
Don't take my word for it, just read the developers blog:
Systemd has made it impossible for me to run an up-to-date Gnome on FreeBSD. That feels a lot like taking away my freedom.
I understand your good motivations but just saying "it will be ok" doesn't make the problem go away. Yes I can write my own init system, that's good. Can I uninstall systemd from most system that install and depend on it by default. Write my own distro? You can't just easily plug and play it.
It is a big like the kernel. Just swapping out Linux kernel with a FreeBSD one doesn't quite work.
> system not baseless accusations that they are making a power grab.
With that I agree. Maybe to get the conversation back on track is to present a few valid technical points in response. Or just ignore it. Saying "hold up people, no fighting please" doesn't work as well in such forums.
I think this is a bit of a stretch. It's not like they did a hostile take over of udev. The maintainers also thought systemd was the right place for that code to live.
As for bloat, there certainly have been some new features, but so much of the systemd code (from what I can tell) was existing code that now lives in one place. That means it's free to consolidate the utility code it uses (every project has helpers for what (g)libc does not provide). In the grand scheme of things, less duplicated code is a good thing.
http://lists.freedesktop.org/archives/systemd-devel/2014-May... (via http://redd.it/2a2tz5):
> Also note that at that point we intend to move udev onto kdbus as transport, and get rid of the userspace-to-userspace netlink-based tranport udev used so far. Unless the systemd-haters prepare another kdbus userspace until then this will effectively also mean that we will not support non-systemd systems with udev anymore starting at that point. Gentoo folks, this is your wakeup call.
So you're complaining about politics, while your only substantive criticism is a purely political one?
There is no fundamental problem that it "solves" which other UNIXes presumably still does have. The problem does not exist. AIX, Solaris, *BSD and many old-school Linux guys will tell you that.
Also, any old-school guy will tell you that a kitchen-sink, put-it-all-in design is a wrong way.
btw, user processes supervision is a task of an OS kernel, which it handles via a bunch of specialized syscalls, not of some "man-in-the-middle" user-level daemons.
There is actually nothing to talk about, except some ambitions and bad designs.
I'm pretty sure /sbin/init runs in userspace even on *BSDs and Solaris, and does process supervision.
initscripts has nothing to do with /sbin/init, surprise?
Let's say a have a program to run - nginx - very popular program. Tell us, please, what improvements the good stuff gives me?
BTW, in my opinion, the SMF has been written "because we can" (use XML for configs and Java for its processing) not because shell init scripts has been broken. There are many strange reasons why some programs were written or has been adopted.
That's pretty close to a no true Scotsman fallacy, isn't it?
The biggest problem I see with system is that the developers don't play well with others. Instead of working with various parties, like the glibc maintainers, to fix deficiencies elsewhere, they expect developers everywhere to drop what they're doing to redesign how their projects work, when they work just fine for the many, many other unix architectures out there. Too much of systemd is based on magical pixie dust, compatibility be damned, and not enough on actually making things better.
And anyway, correctly daemonizing is only part of what systemd got right. I think the fact that you don't need to track services with random programs anymore (since systemd knows each and every process parent thanks to cgroups) is something much more interesting than just getting daemon() right.
That's actually not something you want. It turns out that makes process management unnecessarily hard. That said, the glibc implementation isn't terribly good either. The CW is not to use either.
> I think part of systemd's problem, as much as Poettering et al will try to deny it, is that it is full of NIH.
The most exasperated criticism I see about systemd is its use of dbus for a communications infrastructure, because dbus is both a system bus and a desktop session bus, and everyone associates it with the latter. If they'd just done the NIH thing and rolled their own communications protocol (like various other parties), they'd be deflecting a lot of that criticism.
Honestly, the NIH syndrome seems at least as prevalent amongst systemd's critiques.
[1] http://lists.freedesktop.org/pipermail/dbus/2013-July/015726... [2] http://lists.freedesktop.org/archives/systemd-devel/2014-May... [3] http://lists.freedesktop.org/archives/systemd-devel/2014-May...
So personally as a sysadmin that occasionally runs daemons from the command line I much prefer a double fork(); I don't want the daemon to exit when I log out.
The main thing that scares me is the binary loging format I can think of some benefits but mostly it just seams scary. I guess I will get go se later if the benifits outweighs the rest.
In principal everybody is on terms with the need for a new and modern init system. But yet I'm not even sold on this issue. sysvinit is still holding stance with extra tools and doing it's job cleanly. By introducing a fully reimplemented and still controversial system with many dependencies and with need for many reimplementation on our existing software we are not helping the issue but blurring the waters.
And What's the fascination about boot times?
Nowadays on desktops nobody boots. You just boot once and hibernate/suspend forever. And for servers, if you are rebooting you are doing something wrong. So pulling efforts from building controversial init systems to optimize hibernate/suspend in the kernel would be a better effort on this field.
My take: Containers are not well managed by general, daemon-oriented process supervisors with a localhost-oriented purview. However, those supervisors would do well to use container-related features to better secure and manage daemons as appropriate. In future, processes will be more likely managed across clusters by parallel capable supervisory systems with high availability goals and network infrastructure configuration, load and topology knowledge. Less and less people will even see the init system, except perhaps behind a logo or as it flashes past while booting their device in debug mode.
(Edit: stumbled on http://www.gossamer-threads.com/lists/gentoo/user/284741 which explains the scenario .. would hate to be on BSD)
As far as my experience goes, I've found it actually works really well on all the servers I've moved to CentOS 7 and on the Fedora desktop I play around with (my main dev machines are Macs) it's significantly improved boot time...
I'm sure there are some valid concerns about design and such, but as far as my usage in production goes, I can't say I've had a single problem with it... It makes it a lot easier when I need to write files then the messy init scripts before also.