The high memory usage of ruby still causes problem if the app is single-threaded. I scaled databases for ruby apps for a living for almost 8 years, and sadly single-threaded legacy ruby app is still a thing.
Anyway, in the single-threaded scenario, the app may appear to be CPU bound under the steady state. However, when some hiccup happens in a database or in another microservice, all the ruby processes could soon be blocked waiting for network responses. In this case, ideally there should be plenty of idling ruby processes to absorb the load, but it will be rather costly to do so due to the high memory usage.
There are potential fixes of course, but with trade-offs:
- Aggressive timeout: May cause requests to fail under the steady state
- Circuit breaker: Difficult to tune the parameters, may not get triggered, or may prolong the degraded state longer than necessary. Also not a good fit when the process is single-threaded, as it can only get one data point at a time.
- Burning money: Can only do this until we hit the CPU : memory ratio limit imposed by the cloud vendors.
- Multi-threading: Too late to do this with years of monkey-patching that expects the app to run single-threaded.
