undefined | Better HN

story

0 pointsschneems5y ago0 comments

> GIL

FWIW the GIL has been the GVL since YARV was merged in and it became based on a virtual machine rather than purely interpreted. I believe this was 2.0.

> because the number of copies you run will be proportional to the number of cores you have, not the number of machines

While this is true, Ruby is also very CoW optimized so while forks grow linerally in size (with count), usually the first fork is drastically smaller than the process it was forked from.

I work at Heroku and recommend perf settings to customers. 5 years ago people were mostly hitting memory limits. Now it's pretty common to see apps that are maxing out the CPU well before coming close to ram limits.

Especially when compared to javascript, Ruby is extremely memory efficient.

I agree with your larger statement but wanted to chime in and expand on those two points.

0 comments

jashmatthews5y ago

CRuby could still be much better at CoW. In theory, a forked process only needs a similar memory allocation to a pthread. In practice the runtime writes in a bunch of these inherited pages and fucks it up. malloc-ed memory is usually bigger than the "Ruby heap" so that kind of limits the impact you can have by trying to not write/re-write.

wjossey5y ago

Just want to reaffirm this post. I scaled ruby for a living for almost 8 years to millions of request per minute and this post is 100% accurate.

throwdbaaway5y ago

The high memory usage of ruby still causes problem if the app is single-threaded. I scaled databases for ruby apps for a living for almost 8 years, and sadly single-threaded legacy ruby app is still a thing.

Anyway, in the single-threaded scenario, the app may appear to be CPU bound under the steady state. However, when some hiccup happens in a database or in another microservice, all the ruby processes could soon be blocked waiting for network responses. In this case, ideally there should be plenty of idling ruby processes to absorb the load, but it will be rather costly to do so due to the high memory usage.

There are potential fixes of course, but with trade-offs:

- Aggressive timeout: May cause requests to fail under the steady state

- Circuit breaker: Difficult to tune the parameters, may not get triggered, or may prolong the degraded state longer than necessary. Also not a good fit when the process is single-threaded, as it can only get one data point at a time.

- Burning money: Can only do this until we hit the CPU : memory ratio limit imposed by the cloud vendors.

- Multi-threading: Too late to do this with years of monkey-patching that expects the app to run single-threaded.

jashmatthews5y ago

Dealing with latency variability is a Hard Problem™ and really not much to do with Ruby or process vs thread parallelism.

https://dl.acm.org/doi/pdf/10.1145/2408776.2408794

throwdbaaway5y ago

Well, having more spare ruby processes / threads would make the app more resistant to latency variability, and could have made some incidents into nonevents.

Also, while I don't disagree that it is indeed a hard problem, I do have very good experience with an async java stack, where I didn't have to worry about things like this. As long as a sane queue limit is defined on let's say the jetty http client, if something bad happens at the other end, the back pressure would kick in by failing immediately the requests that couldn't make it into the queue. Other parts of the app would then continue to be functional.

So, I would contend that it has a lot to do with ruby high memory usage, made much worse when single-threaded, and it looks like ruby 3.0 still won't have a complete async story yet?

EDIT: I checked the link again, and it looks Jeff Dean was talking about latency at p999 or above? By "hiccup", I actually mean something that would increase avg latency by perhaps 5~10x times, e.g. avg latency of 100ms under steady state + timeout of 1 second + the remote being down. Sorry for the confusion. Here, I am lucky if people start caring about p95.

1 more reply

j / k navigate · click thread line to collapse

0 comments

jashmatthews5y ago

wjossey5y ago

Just want to reaffirm this post. I scaled ruby for a living for almost 8 years to millions of request per minute and this post is 100% accurate.

throwdbaaway5y ago

There are potential fixes of course, but with trade-offs:

- Aggressive timeout: May cause requests to fail under the steady state

- Burning money: Can only do this until we hit the CPU : memory ratio limit imposed by the cloud vendors.

- Multi-threading: Too late to do this with years of monkey-patching that expects the app to run single-threaded.

jashmatthews5y ago

Dealing with latency variability is a Hard Problem™ and really not much to do with Ruby or process vs thread parallelism.

https://dl.acm.org/doi/pdf/10.1145/2408776.2408794

throwdbaaway5y ago

Well, having more spare ruby processes / threads would make the app more resistant to latency variability, and could have made some incidents into nonevents.

So, I would contend that it has a lot to do with ruby high memory usage, made much worse when single-threaded, and it looks like ruby 3.0 still won't have a complete async story yet?

1 more reply

j / k navigate · click thread line to collapse