freopen(NULL, "w", stdout);I mean, it's cute that it's in principle possible, but what does it actually do that can't be achieved more cleanly in other ways?
Specifically, the case of freopen when the path argument is null, which was introduced in C99.
freopen does all the open-time actions such as that the "w" mode truncates the file. So that could be why. If we dup the descriptor, we have to call ftruncate to implement the "w" flag. I think if we just open the /proc item, O_TRUNC will do it for us.
The freopen function is wacky in the first place, and obviously the approach has the flaw that (as we learn from the submitted article) that sockets cannot be opened this way.
Obviously, /proc/.../fd was not designed for freopen.
/proc/<other-than-self>/fd is very useful for implementing, e.g. process substitution in Bash:
$ diff -u <(this command) <(that command)
/proc/self/fd/ exists because there is a /proc/self symlink to /proc/<self-pid>/ not necessarily because it's useful for a process to open its own descriptors this way. Other /proc/self things are useful, like /proc/self/exe to find out where your executable is located.
Well you can't `dup()` an fd that you don't already have, this would let you do it. Assuming I understand the behavior right it doesn't have to be the original program calling `open()`.
If you're running a separate process and trying to debug another process that's running by peeking at its fds, then being able to just `open()` them is easier and less invasive compared to the alternatives for getting a dup'd fd (Ex. force the program to call `dup()` itself and then send the fd over a socket or something).
Also known as file handles referencing file objects (on Windows). Unix terminology is unnecessarily confusing in this place IMHO.
On the subject of the article itself: why was this change introduced? To give kernel support for reopen(3) since dup(3) already exists?
Honestly, the OS situation is a mess and I just want to not think about it, but not having an OS is not really an option.
That we use the term "file descriptors" for pointers from userspace to any kernel object, even those that are not files, is unfortunate, but ultimately just a naming quirk. Windows has a better name, "Handle", but the concept is exactly the same.
The OS includes the file system, and file systems include a notion of paths, and relative paths are really useful. So, the OS helps you by automatically resolving relative paths to your current directory, instead of forcing every application to manually keep track of this.
Linux is perhaps the only popular OS whose interface is not defined in C. All syscalls are clearly documented at the assembler level in Linux, and kept backwards compatible. All other popular OSs (Windows, MacOS, FreeBSD) have a C lib you have to dynamically link if you expect compatibility.
Even if signals weren't a thing, you'd still have to worry about processor interrupts. There is no such thing as a purely single-threaded program on any gpCPU released in the last 30+ years.
The variety of calls in Linux to handle various kinds of events is unfortunate. Windows has a slightly cleaner interface, though even there it's not ideal. Hopefully io_uring will subsume all of the current use cases.
The numbers after the syscalls are related to the man pages where they are documented. Not all that relevant.
Sycalls are not functions, they are specific APIs that the kernel provides to userspace, defined at the assembler level (you put this value in this register/stack and jump to this address/invoke this CPU interrupt). It is up to your language to wrap syscalls into functions, which may have an entirely different calling convention. A kernel can't provide APIs as language-specific functions, as Python's calling convention is vastly different from Haskell's.
Fork() has many meanings that are not related to cutlery, used in CS in other places. Fork() is also an extraordinarily terrible interface for process creation for reasons which have nothing to do with its name. I would be happy if one day Linux gets rid of this insanity and adds a CreateProcess syscall that doesn't have to pretend to copy the entire address space of the current process.
fork() is going to exist forever, but posix_spawn() already exists:
However, this still means that your process will be affected by any memory correctness issues in "libsyscall", in addition to the issues in the kernel itself. Plus, the maintainers of libsyscall would have to write it in a bizarre dialect of C that doesn't use any stdlib functions, which might price even more error prone than standard C.
It's perhaps important to note here that the parts of libc that implement syscalls in OpenBSD are not simple syscall-to-C wrappers, they can have quite a bit of code occasionally. And Windows' runtime library is even more complex than that. That's their whole point - they can keep a backwards compatible system interface in spite of significant changes at the syscall layer, probably by doing lots of small pieces of work in userspace to bridge the gap.
One is that fork() is by definition a very costly operation (a copy of the entire address space of the current process), and the kernel has to do a lot of work to implement it efficiently (implementing copy-on-write clones of all of the pages of a process). And that all that work is done for nothing in the very very very common case of doing fork() + exec().
The other problem is that the semantics of fork() just fundamentally can't work properly for a multi-threaded process. In any multi-threaded process, if you do fork(), the only thing you can safely do in the child process is to call exec(). Any other call, even a printf() or some path logic, has a very good chance to lead to a deadlock, quite possibly inside malloc() itself.
So fork() as a standalone operatikn is actually an extremely niche utility (duplictaing single-threaded processes) that has been made the main way of spawning new processes. Similarly, exec() by itself is an even more niche utility, sometimes useful for "launcher" style processes.
So, instead of achieving an extremely common task (launch some binary file as a new process) using a dedicated system call, Unix has chosen to define two extremely niche syscalls that you should almost never use individually, but that together can implement this common process, but only with a lot of behind the scenes work to make it efficient.
The first is to implement POSIX shells, and that's less because this is a good design and more because shells are a wrapper around the original Unix system calls. Note that if you're designing a scripting language that isn't beholden to compatibility with /bin/sh (especially one that can be portable to OSes that don't have fork()!), then you're liable to not design it in such a way that requires you to use fork().
The second use case is an alternative to threads for parallel processing. And there are some reasons that processes can work better than threads for parallel processing. But fork() has such a bad interaction with multithreaded code [1] that you end up having to choose fork() xor threads. And as threading has become an increasingly important part of modern environments, well, given that xor choice, almost everybody is going to come down on the threads side of the equation.
The final use case, and by far the most common, is to be able to spawn a new process. This means you break up one logical system call (spawn) into two (fork + exec), the first of which semantically requires you to do a lot of work (clone memory state) that you're immediately going to throw away. Even in the case where you want to do more expansive process-twiddling magic before spawning the process, there are better designs (especially if you're willing to commit to a handle-based operating system).
Of the three use cases, one amounts to "backwards compatibility", and the other two amount to "fork() is actively fighting you". That is not the hallmark of a good API.
[1] Think things like "locks are held by threads that don't exist."
https://news.ycombinator.com/item?id=19621799 180 comments
https://news.ycombinator.com/item?id=22462628 117 comments
https://news.ycombinator.com/item?id=8204007 314 comments
https://news.ycombinator.com/item?id=31739794 135 comments
https://news.ycombinator.com/item?id=16068305 89 comments
that's some 1000+ comments for light background reading about fork(). I think you can make some aggregate sentiment analysis from that and conclude that fork() is not great.
Also some of your specific concerns are impossible to resolve. Take SIGNALS for example. They're ostensibly just callback functions / events. It's very easy to do event-driven programming if all of your events are being raised by the same language runtime as the code you're writing your application in, but how do you raise an event that crosses application boundaries and where your application code would be written in a different language to the event bus (the kernel in this instance)? You ultimately end up with some kind of IPC ugliness and the best solution in the 70s was SIGNAL. Given its now core to the OS, stripping out SIGNALs from Linux would be as easy as stripping HTML from websites.
There are plenty of radically different successors to Linux though. But they all have their own rough edges too. Ultimately these things are complicated and you always end up making compromises somewhere.
You can program in another language other than C and avoid using GNU libc, or Musl libc, or any other libc, so avoid using the C API to talk to the kernel. Other languages like Rust and Go provide their own runtimes and avoid using the C runtime for syscalls. Syscalls are written in assembly language, or at least syscall(2) itself is because the kernel API is just marshalling and a context switch. You misunderstand.
Oh, and the fork(2) function on Linux is implemented by a libc using the clone syscall(2). The (2) is the chapter in the manual providing the documentation. You misunderstand.
> You can program in another language other than C and avoid using GNU libc
Linux is a set of various system interfaces. Many of which are exposed through libc functions. If you deliberately exclude libc and program something parallel to it instead, then you aren't really using Linux. You've created a hybrid system. This only reinforces OP's complaint about not being able to program in a different language, because, in other words, this means that to C language programmer Linux is available fully, and to other languages the functionality is not completely available.
Take, for example, async I/O, which is implemented in libc around other system primitives, s.a. threads. If you don't use libc, you don't have async I/O, and by extension, you don't have Linux, since Linux is supposed to have that.
Linux as a project is the kernel. Unlike most OSes, the most commonly used libc is maintained separately. As a result, Linux the operating system is the one where it’s the most common and supported to avoid the “standard” libc because the kernel itself has a stable interface.
For example, here is a site that documents all Linux syscalls on x86_64, with the exact registers that they read their arguments from, and the registers they write their results in:
Legitimate Linux applications are written in Go all the time. Linux applications are often written in Rust and use the Linux ABI without going through any libc. They seem to be doing mighty fine in terms of "being Linux".
This sounds like some variant of the True Scotsman fallacy.
cpu% ./dup -proc > out; cat out; echo
1d
cpu% ./dup -dup > out; cat out; echo
1d
The slightly modified code can be found here: http://okturing.com/src/18632/body
This seems to be another case of Linux attempting to emulate Plan 9 design and not quite hitting the mark. In general these operating system interfaces are much more consistent and sane within the Plan 9 environment at (seemingly) every turn. I think a lot of folks see the implementation quirks of Linux and decide its time to toss the baby out with the bath water, but studying Plan 9 really shows how nice things could have been.
What do you think about io_uring? I believe it moves away from OS interface as (C) function calls.
> Processes are created in Linux in an especially simple manner. The fork system call creates an exact copy of the original process.
> The Linux sequence of clone plus exec is yet more orthogonal, since even more fine-grained building blocks are available. As a general rule, having a small number of orthogonal elements that can be combined in many ways leads to a small, simple, and elegant system.
Yup, still the same bullshit. It is not elegant, it is not fine-grained, it is not simple, it is certainly not small.