undefined | Better HN

0 pointsistvan__11y ago0 comments

on this argument you could state that Erlang's processes the way to go because OS threads are way more expensive. If I have a problem that I can easily make parallel than I could just start up N OCaml processes and send each process a chunk of work and this would not be much more inefficient than the thread based implementation. On the top of that, I don't like to create a tonn of threads in any application, I much rather have a fixed number of threads and using channels to send them work than dynamically (on demand) creating threads or processes.

0 comments

2 comments · 1 top-level

bad_user11y ago· 1 in thread

Erlang's processes are actually better than threads for many use cases, yes. But the thing is that with 1:1 multi-threading you can build many abstractions on top and for example on top of the JVM frameworks using Erlang's model like Akka [1] and Quasar [2] are very popular. Either way, you can't compare Erlang with platforms whose notion of parallelism involves POSIX's fork, but lo and behold, I'm comparing it with Java because I can.

MOST problems are not easily parallelizable and you'll end up with concurrency. Concurrency means synchronizing and sending messages between processes. For example sending messages between processes means opening a socket of some sort, serializing the data you want to send and deserializing it on the consumer's side. That's extremely inneficient and there's no way you can end up with a pipeline of tens of millions of messages per second, but with 1:1 multi-threading you can [3]. Erlang can't cope with this load btw.

One other thing I like about working with threads is the user-friendliness. Yes, skipping over the perils of multi-threading which you can sort of avoid by using better libraries, you can easily do things like number crunching using "parallel collections", combine actors with reactive streams and futures, or fake asynchronous I/O by blocking threads. People nowadays tend to underestimate the utility of blocking threads, but it's pretty cool having an interface like `def fetchData: Future[Result]` that could be implemented on top of Netty (asynchronous I/O) or with JBDC (blocking I/O), only suffer a very small penalty and your process to still be able to reach 80% of CPU utilization.

So I think OCaml getting multi-threading support is a pretty big deal.

[1] http://akka.io/

[2] https://github.com/puniverse/quasar

[3] https://lmax-exchange.github.io/disruptor/

istvan__OP11y ago

Thanks. I think Erlang is the grandfather of languages using the actor pattern and at the same time Joe realized that POSIX threads does not belong to business logic code and it can be done without leaking the number of actual OS threads to the user's codebase. On the top of that Joe also implemented the message passing to avoid the shared memory that is a serious problem from the safety point of view for applications with threads. One thread can take down the entire process execution. Addressing these in Erlang made it possible to write the first commercial system with extra high availability and fault tolerance.

JVM is obviously great a VM, and Scala brings most of the Erlang (and many other languages) features to the table. I am not sure about the fault tolerance. I need to look into how Akka implements the actors.

> Concurrency means synchronizing and sending messages between > processes.

I think in Erlang you can do only async sending and when the receiving process wakes up it gets it through one of the means you mentioned.

http://bartoszmilewski.com/2009/02/10/message-passing-sync-o...

> Erlang can't cope with this load btw.

This is what I am curious about. I have seen only one big system written in Erlang that was massive big. WhatsApp was also running in Erlang and they achieved something like 1M connection/server. That was impressive. I am wondering how could you do that with Akka?

> https://github.com/puniverse/quasar

I am following the development of these, starting to use it in Clojure soon, I am curious how it works out. Personally I prefer Aleph on the JVM for Clojure projects but wanna see what is happening in Quasar & co.

https://github.com/ztellman/aleph

j / k navigate · click thread line to collapse