undefined | Better HN

0 pointsjwatzman11y ago0 comments

> The second option is to do an oldschool fork() and communicate with message-passing. Or have a master that manages workers and communicates with ZMQ, HTTP, TCP, IPC, etc. Or use a library that does it all for you like parmap, Async Parallel, etc etc.

I work on the Hack language typechecker at Facebook. The typechecker is written in OCaml, and since it needs to operate on the scale of Facebook's codebase (tens of millions of lines of code), it's a pretty performance-sensitive program. We needed real parallelism, but doing it with fork() and IPC was too costly for us, both in terms of storage (if you aren't careful you end up duplicating a bunch of data) and CPU (serializing/deserializing OCaml data structures to send over IPC is CPU-intensive).

We ended up doing something somewhat more interesting. Before we fork(), we mmap a MAP_ANON|MAP_SHARED region of memory -- that region will be backed by the same physical frames in each child after we fork, so writes to it in one child process will be visible in the others. We use a little bit of C code to safely manage the shared-memory concurrency here.

The code for this all open source (along with the rest of the typechecker, HHVM runtime, etc) if you want to take a look: https://github.com/facebook/hhvm/blob/master/hphp/hack/src/h...

I also gave a tech talk a while ago on internals of the type system and typechecker; the latter part starts here: https://www.youtube.com/watch?v=aN22-V-b8RM&feature=youtu.be...

0 comments

3 comments · 1 top-level

AceJohnny211y ago· 2 in thread

> We ended up doing something somewhat more interesting. Before we fork(), we mmap a MAP_ANON|MAP_SHARED region of memory -- that region will be backed by the same physical frames in each child after we fork, so writes to it in one child process will be visible in the others. We use a little bit of C code to safely manage the shared-memory concurrency here.

Isn't that similar to how Linux implemented threads for a long time (before NPTL [1]) ?

I vaguely recall that for a long time people were complaining about the cost of starting threads in Linux, because it basically amounted to fork()+shared memory.

[1] http://en.wikipedia.org/wiki/Native_POSIX_Thread_Library

jwatzmanOP11y ago

I don't know the history of threads/NPTL on Linux. However, the distinction between "thread" and "process" in the Linux kernel is mostly a human one, not a technical one. Take a look at the clone() syscall -- spawning a thread vs. forking a process amount just to different flags to that call, to tell it whether to copy pages or not, how to assign a new ID to the new thread/process, etc. (Not sure if that's how fork() and friends are actually implemented under the hood.)

cbd198411y ago

> (Not sure if that's how fork() and friends are actually implemented under the hood.)

If you use strace you can see that it is. fork(2) and pthread_create(3) both show up as calls to clone(2).

1 more reply

j / k navigate · click thread line to collapse