undefined | Better HN

0 pointsjerf6y ago0 comments

"Would it have been better if they went with Elixir?"

No. It would have been unshippably bad. BEAM is generally fairly slow. It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. As a language, it is much slower than Rust. Plus, if you tried to implement a gigantic shared cache map in Erlang/Elixir, you'd have two major problems: One is that you'd need huge chunks of the map in single (BEAM) processes, and you'd get hit by the fact BEAM is not set up to GC well in that case. It wants lots of little processes, not a small number of processes holding tons of data. Second is that you'd be trading what in Rust is "accept some bytes, do some hashing, look some stuff up in memory" with generally efficient, low-copy operations, with "copy the network traffic into an Erlang binary, do some hashing, compute the PID that actually has the data, send a message to that PID with the request, wait for the reply message, and then send out the answer", with a whole lot of layers that expect to have time to make copies of lots of things. Adding this sort of coordination into these nominally fast lookups is going to slow this to a crawl. It's like when people try to benchmark Erlang/Elixir/Go's threading by creating processes/goroutines to receive two numbers and add them together "in parallel"; the IPC completely overshadows the tiny amount of work being done. (They mention tokio, but that's still going to add a lot less coordination overhead than Erlang messages.)

Go is a significantly better language for this use case than Elixir/Erlang/BEAM is, let alone Rust.

(This is not a "criticism" of Erlang/Elixir/BEAM. It's an engineering analysis. Erlang/Elixir/BEAM are still suitable for many tasks, just as people still use Python for many things despite the fact it would be a catastrophically bad choice for this particular task. This just isn't one of the tasks it would be suitable for.)

0 comments

3 comments · 3 top-level

hopia6y ago

Not to disagree with your analysis of the performance implications, but I don't think having all that data under a single or a few processes would be the right architectural pattern to handle this in Elixir.

The article says that the data is basically "per-user", indicating that the active client connection process could be used to store the data. It already hosts other data related to the client (connection) anyway. I think updating and querying it globally would be the trouble in that case.

Another could be storing the data in mnesia, BEAM's internal mutable in-memory DB. Probably better, but still not ideal to solve this.

Anyway, you're right in that no matter how you'd try to solve this problem on pure Elixir you'd still be seeing some bottlenecks because BEAM just isn't very well suitable for this kind of problems, hence Rust.

But can you elaborate on what you mean by other platforms catching up with Elixir's inherent concurrency advantages? Which modern platforms give similar features?

1 more reply

blackrock6y ago

Good points.

> It wants lots of little processes, not a small number of processes holding tons of data

Elixir/Erlang is good for handling a lot of little processes with a small amount of data. And not for a small number of processes, handling a large amount of data.

The little processes holds smaller data, and it just gets dropped, after the function is done, instead of getting reclaimed by a garbage collector.

This is probably what makes Elixir/Erlang good for telecom equipment, like packet switching hardware, but not good for more complex software applications that may need to fetch and manipulate a lot of structured data in multiple stages.

In this case, does anyone know of Elixir’s maximum throughput?

sergiotapia6y ago

>It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020.

Such as?

1 more reply

j / k navigate · click thread line to collapse