undefined | Better HN

0 pointstoast019d ago0 comments

The zygote pattern[1] is a great optimization to deal with the cost of forking, but IMHO, being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better.

I would guess it would be a small difference in measurable performance between zygote and a direct clean spawn, but it's one less trick an application needs to do, and it would be very helpful for libraries that spawn things. Spawning inside a library isn't always a great thing to do, but some things would really benefit from process level isolation.

[1] In case one isn't aware, the zygote pattern involves forking a 'zygote' process during application startup, and having that process do any forks that need to happen during application runtime. This reduces the cost of forking in large applications, because the zygote will have few fds open and use little memory. This lets your large application spawn new processes without delaying the application or the startup of the new processes. Some applications will spawn many zygotes to allow parallelism for spawning at runtime.

0 comments

8 comments · 2 top-level

pizlonator19d ago· 3 in thread

You're referring to something else, and maybe I'm using the term "zygote" incorrectly.

In all uses of zygotes that I have seen, here's what's really happening:

- `fork` is being used to reduce the cost of starting a process that has a high start-up cost. So, you start one process, run it through the expensive initialization, and then fork it from there to start new processes.

- To make this even faster, you have a pool of pre-forked processes sit around.

- Having pre-forked processes sitting around ready to be used is not expensive because of the CoW property and the fact that a process that forks and then immediately pauses will not have triggered any significant CoW yet.

So, the zygote optimization you speak of is in practice only meaningful on top of systems that are using an optimization uniquely enabled by `fork` (avoiding process initialization costs by cloning a process), and that zygote optimization is further optimized by another property of `fork` (memory sharing of forked processes that haven't done anything else yet).

toast0OP19d ago

Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says:

> A zygote process is one that listens for spawn requests from a main process and forks itself in response. Generally they are used because forking a process after some expensive setup has been performed can save time and share extra memory pages.

I think reading the first sentance and stopping covers my zygote, but adding the second sentance covers yours. So I think we're both right!

I think both paths are useful. If your children need time to startup and become ready, spawn one that does start up work, and then it (pre)forks at the ready state to have processes ready to handle requests (your zygote). This does require a traditional fork() to avoid duplication of work.

But if forking is expensive at runtime because you have a million FDs open and a whole lot of memory allocations, spawn spawners before you start doing work (my zygote). This could be unnecessary with a inexpensive way to spawn a new process from an process that has lots of resources in use.

Of course, you can also use my zygotes to spawn your zygotes. Zygoteception.

[1] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...

skydhash19d ago

I quite like the idea. I’m using OpenBSD on an oldish laptop, and fork-exec is expensive enough that it conflicts with the usb subsystem. Isochronous transfers have a 1ms realtime requirement and it seem that the fork-exec system calls hold the giant lock long enough to mess with it (audio stutters).

While I’ve not bothered to profile it, but it seems that process that have lot of mapped pages is the issue (firefox, emacs,…). In the emacs case, the issue is when the main process trying to fork-exec, if I start a shell session (with shell-mode or term-mode), it works fine.

mpyne18d ago

> Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says:

Google may have popularized the term, but this approach was already in use by KDE developers in the KDE 2.x timeframe, where it was used as part of a system called kdeinit.

In this scheme, launching KDE apps from a KDE desktop could bypass much of the startup cost of dynamic linking by forking from a long-running kdeinit process (with kdeinit itself deliberately linked to all large dependency libs like Qt and kdelibs), dynamically loading the application logic (stored as a .so) and then launching the app.

This was more to save startup time due to how long it took to dynamically resolve a multitude of C++-based symbols back then, all the common logic came before the app's own main() would ever be called. But it did also save a bit of memory as well.

PaulDavisThe1st19d ago· 3 in thread

> being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better.

It's called clone(2)

toast0OP18d ago

adding on the the sibling, what argument to clone allows me to set the fds of the child? AFAIK, you either share the FD table with the parent, or get a copy of it. If the parent has 1 million FDs open and the child doesn't want most of those, dealing with that has real costs. Many applications that tend to have large numbers of FDs and also fork/exec will mitigate the cost by spawning a process during startup that they can then use to spawn processes during runtime without doing it from the main process; this is a nice mitigation, but it shows a missing interface.

trumpdong18d ago

Which argument to clone starts the process with an empty address space?

eggnet18d ago

That happens with execve(). clone() allows you to not copy the page table prior to the execve() call.

1 more reply

j / k navigate · click thread line to collapse