Specifically for the request case, I recall seeing a talk suggestion that processing requests last-in-first-out resolves this failure mode as well as reduces the 99th percentile latency. The intuition is if a request is going to be late, might as well abandon it and process it later because it's already late instead of doing a slow job and making everything else slow in the process.
Maybe that's a strange metaphor for work too.
How does it manage to do that?
When you're getting close to your limits and requests are actually waiting in the queue, I would expect FIFO to slow down everything to provide backpressure, while LIFO keeps a very nice median speed but has an increasing percentage of requests that timeout and retry once or even twice.
Are there significant dynamics I'm not thinking of?
Maybe if you have bursts that are just big enough for FIFO to delay >1% of total requests, but small enough for LIFO to drop <1%? But in that situation giving a single percentile paints a misleading picture; 99th would do better on LIFO but it comes at the cost of trashing higher percentiles.
LIFO processing in the steady state is not a great idea because it will stochastically starve some requests for no reason.
Requests spikes cause the whole system to backup where appropriate upper limit tuning allows the servers to drop requests before they get overloaded and grind to a halt
