I suspect he was referring to socket activation and how that simplifies these kinds of messes.
> Nor should they. There are a ton of ways to skin this cat in userspace, depending on what your application needs. Protobufs come to mind, for example, but there are others.
Right... so that's exactly what systemd did. It used Dbus, which provides that standard serialization format. Not my favourite format, but very well established and tested and focused on systemd's problem domain.
The point is, in order to have loose coupling between components, something like unix pipes is just a starting point.
> Does the pipe know what's best for every single application that will ever use it?
Ah, now I understand the problem with systemd. I never realized it was trying to take over every application's communications protocol! ; -)
Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of...
> Last I checked, you pass file descriptors via UNIX sockets, not pipes.
Correct. People have a tendency to mess up their semantics though. If the original poster wasn't referring to unix domain sockets, than it is an even sillier question.
> It is unreasonable to expect the pipe to be aware of every single threat model an applications expects, especially since they change over time.
Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components.
> Nor does DBus. Nothing stops an application from willy-nilly changing the data it serves back.
? D-Bus will drop you like a hot potato the moment you fire off invalid messages. You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up.
It's also an ending point. If each application gets to define its own IPC primitives, then there become as many app-to-app communication protocols as there are app-to-app pairs. This does not make for a loosely-coupled ecosystem.
> Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of...
This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol.
> Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components.
Um, data within a pipe is visible to only the endpoint processes, the root user (via procfs and /dev/kmem), and the kernel. If you don't trust an endpoint, you should stop communicating with it. If you don't trust the root user or the kernel, you can't really do anything securely at all in the first place. My point is, data within a pipe is about as secure as it's going to get.
> You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up.
I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it. DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using.
That's exactly why you need a more systemic approach to the IPC mechanism...
You can pretend that "oh this is just a stream so there isn't tight coupling", but the information that is communicated is the same. If you haven't imposed some structure and consistency to it, that's exactly how you end up with a ball of mud.
> This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol.
You misunderstood my point. I'm not justifying it on the basis that systemd is using it. I'm saying the fact that all the other systems have arrived at a similar, and in many cases the exact same mechanism, is pretty strong evidence that it is a reasonable design choice.
Basically all the Linux systems out there are already using udev & dbus. Most of the non-Linux systems do as well. Everyone's done it and made it work. That systemd is adopting arguably the most entrenched one in the Linux sphere is hardly as controversial as people seem to think it is.
> My point is, data within a pipe is about as secure as it's going to get.
I wasn't trying to suggest it wasn't a secure point-to-point communication mechanism (it has issues, but fine enough). The issue is that you need more integration in with the security model to avoid having a rats nest of security logic on top of it.
> I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it.
There's a very long and glorious history of malformed and misleading IPC causing problems. Not that it is the only thing, but life becomes a lot easier when that problem is off the table.
> DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using.
Yes you will. However, DBus ensures that you don't have to write a ton of redundant code just verifying you are getting validly structured data and dealing with the nasty ways someone might try to exploit that.
Imagine having to write a secure REST service where the only thing you had to worry about in the entire network protocol stack was the validity of the data expressed in the payload.
So, you basically want to turn IPC into CORBA. It's a bad idea to have the OS impose too much structure on your IPC, in the same way that it's a bad idea to have the base class in an object hierarchy try to take on too many subclass-specific responsibilities. This is because over-specialization of a component needlessly constrains the designs of systems that use it.
That said, you are correct in that byte streams alone do not make for loosely coupled systems. Programs must additionally emit data such that other unrelated programs can operate on it without modification. But we already have this universally-parsable data format: it's called human-readable text. It's why you can "grep" and "awk" and "sed" the outputs of "ls" and "find" and "cat", for example.
Take a second and imagine what the world would be like if you had to write "grep" such that it had to be specifically designed to interact with "find," instead of simply expecting a stream of human-readable text. Imagine if "awk" had to be specifically designed to interact with "ls." This is the world that CORBA-like IPC creates, where programs not only need to be intrinsically aware of the higher-level RPC methods each other program exposes, but also intrinsically aware of the access and consistency semantics that go along with it. No thank you; I'll stick with pipes and human-readable text, where the data format, data access, and consistency semantics are universally applicable.
> Basically all the Linux systems out there are already using udev & dbus. Most of the non-Linux systems do as well. Everyone's done it and made it work. That systemd is adopting arguably the most entrenched one in the Linux sphere is hardly as controversial as people seem to think it is.
First, udev is Linux-specific--it uses netlink sockets to listen for Linux-specific hardware events. Second, not everyone uses udev and dbus. mdev, smdev, eudev, and static /dev are also widely used and have well-defined use-cases that udev does not serve, and plenty of servers (and even my laptop) get along just fine without dbus.
Trying to justify udev and dbus because "everyone uses them so you should too!" is not only an example of the bandwagon logical fallacy, but also reveals your ignorance of and insensitivity to other users' requirements.
> The issue is that you need more integration in with the security model to avoid having a rats nest of security logic on top of it.
I said it above and I'll say it again here. The IPC layer does not know and cannot anticipate the security needs of every single application. If you try to design your IPC system to do this, you will fail to encompass every possible case. This is because threat models are not only specific to the application, but also specific to the context in which the application runs.
For example, you do not send your bank account password over an out-bound socket unless it has first been encrypted using a secret key known only to you and your bank. Your reasoning implies that the IPC system should be tasked with automatically enforcing this constraint, among others. Nevermind the fact that the IPC system will see only the ciphertext and thus will not know that the data it's about to send contains your password.
> However, DBus ensures that you don't have to write a ton of redundant code just verifying you are getting validly structured data and dealing with the nasty ways someone might try to exploit that.
So do plenty of stub RPC compilers and serialization libraries that have been around longer, are more widely used, and are better tested than DBus. However, neither DBus nor any of these solutions will help you with well-formatted but invalid data. Your application has to deal with that, since the validity of data is both application-specific and context-specific (so, not something the IPC system can anticipate).
Again, what's so special about DBus, besides the fact that it's the New Shiny?
> Imagine having to write a secure REST service where the only thing you had to worry about in the entire network protocol stack was the validity of the data expressed in the payload.
If I'm writing a secure service of any kind, you can be damn certain I'm thinking about a LOT more than input validation! Security encompasses waaaaaaaaaay more than that.
Even if security wasn't an issue, there is still a LOT more to worry about than input validation. Things like scalability, fault tolerance, concurrency, and consistency come to mind. There is no silver bullet for any of these, let alone an IPC system that solves them all at once!
No, very much not, because we don't really need an RPC mechanism here. We want something in the event/messaging space.
But it isn't even a want. If you don't have it, you end up with each of the your components actually being very tightly coupled to all the other components it talks to and you've got a truly monolithic mess on your hands.
> It's a bad idea to have the OS impose too much structure on your IPC, in the same way that it's a bad idea to have the base class in an object hierarchy try to take on too many subclass-specific responsibilities. This is because over-specialization of a component needlessly constrains the designs of systems that use it.
This isn't exactly a new concept or a new problem. There are plenty of existing cases where this is happening (basically every platform I can think of right this moment, though I'm sure there are plenty of exceptions), including in the current Linux udev mechanism.
> But we already have this universally-parsable data format: it's called human-readable text. It's why you can "grep" and "awk" and "sed" the outputs of "ls" and "find" and "cat", for example.
/me falls out of chair.
Yeah, that's worked out great. Never had a problem with init scripts not extracting the right column or handling a new variant in how the output comes out (or even better still, the dreaded "value with embedded whitespace").
But you know what? DBus is basically human readible with a bit more imposed structure than generic streams. So, I think you are arguing in support of the systemd approach without realizing it! ;-)
> First, udev is Linux-specific--it uses netlink sockets to listen for Linux-specific hardware events. Second, not everyone uses udev and dbus. mdev, smdev, eudev, and static /dev are also widely used and have well-defined use-cases that udev does not serve, and plenty of servers (and even my laptop) get along just fine without dbus.
Very true. I was speaking in generalities. Point being, udev is out there and very thoroughly established as something that people seem to generally want.
> The IPC layer does not know and cannot anticipate the security needs of every single application. If you try to design your IPC system to do this, you will fail to encompass every possible case.
You have some ambitious notions for the systemd project that go well beyond the goals that critics say are overly broad in scope. This is for addressing a relatively narrow set of problems that wouldn't even come close to defining 1% of IPC on a LInux system. I'm not suggesting we replace the entire Unix toolset with a complete new set of interfaces and programs (nor are the systemd guys). This is specifically for managing the interactions between devices & daemons... It's a well established problem domain with some well established roles & responsibilities and some pretty well understood data message/event structures.
While it might use systemd/dbus/whatever to get notifications about various services and system events, YOUR BANKING SOFTWARE IS NOT SUPPOSED TO USE SYSTEMD TO MOVE MONEY BETWEEN YOUR ACCOUNTS!
> Again, what's so special about DBus, besides the fact that it's the New Shiny?
DBus isn't the new shiny. It's the old shiny. The new shiny would probably be 0mq or some of the new datagram protocols that people are experimenting with, along with various extensible binary protocols like MessagePack and Cap'nProto.
What's special about DBus is that it is already being used very broadly on Unix platforms for this kind of function and is well integrated in to the system security model. The one bit of additional coolness it brings to the table is the support for socket activation, which simplifies the complexity of start ordering and discovery tremendously, which is indeed a VERY nice benefit, but could no doubt have been NIH'd independently.
> If I'm writing a secure service of any kind, you can be damn certain I'm thinking about a LOT more than input validation! Security encompasses waaaaaaaaaay more than that.
Yes, but the point is one derives substantial benefit from trusted components in the system that take care of their part of the problem. You don't benefit from having to reimplement an entire security apparatus with each component. This is basic security compartmentalization 101.
> Things like scalability, fault tolerance, concurrency, and consistency come to mind. There is no silver bullet for any of these, let alone an IPC system that solves them all at once!
You appear to be simultaneously claiming there is no silver bullet and being terribly upset that systemd isn't one.
Yes, it is no silver bullet. It's not even the huge sea change that some people seem to think it is. Rather, it is an incremental improvement over existing practices that gets rid of a bit more of the cruft and stupidity in the existing infrastructure. Doing that kind of thing right can really make a big difference for the system as a whole, but it isn't the apocalypse.
I was thinking about this comment, and I realized this is probably the source of most of your angst, which leaves some great solutions on the table.
systemd isn't really creating a much more significant break with the systems you like, because it's building on top of what Linux, which for the most part has already made the break.
The problem is, projects like GNOME, which have software you want to use, are integrating more tightly with Linux and specifically bits of systemd.
I think the obvious solution is a bridge/better interface. The contracts that GNOME is going to rely upon are at least going to be pretty well defined, and if you've got another system that works better, it shouldn't be hard for it to provide an equivalent, even compatible, interface.
If it really is demonstrably better, GNOME and other projects will likely adopt your interface/abstraction, and systemd will end up having to communicate through your interface. Even if they don't, it is a comparatively simpler effort for a software community to support a relatively small set of touch points that they want GNOME to be aware of, and maintaining a fork or compatibility layer is a perfectly reasonable solution (indeed, BSD already does this for Linux runtimes).
I can understand why it'd not be a perfect solution from your perspective, but if a bunch of developers contributing to a work you care about are going a direction you don't like, it's about as good an outcome as one could hope for.
