and subsequent enhancement from Yandex folks https://github.com/kndrvt/mhs
Explanation is at https://habr.com/ru/companies/yandex/articles/858662/ use your favorite translate site.
I understand the complexities of having a large set of distributed services behind load balancers, I just think there could be a better way of choosing a backend based not only on least requests, TTFB and an OK response from a health check every N seconds.
You have stick tables and a very rich way of populating them and then you can use these tables of in RAM data to make routing decisions.
Sometimes you need another proxy too - eg Apache/nginx or whatever, perhaps for authn/authz.
Yes it is a tricky concept and this series of articles merely scratches the surface. Good effort though.
Glad you found it a good effort, and I agree there’s room to go deeper in future posts.
algorithms is pretty hard as a spelling: its derived from something like Al Gorism - the name of an Arab chap who documented an early notion. By the time English has decided to create a word, you can be sure it will be ... painful!
Keep going mate, you have a great writing style and presentation.
use DSR and 50% of your traffic is taken care of. https://www.loadbalancer.org/blog/direct-server-return-is-si...
explore load balancing lower in the stack based on ASN to preroute stuff for divide and conquer. (geolocated, etc...)
weighted load balancing only works for uniform traffic sources. youll need to weight connections based on priority or location, backend heavy transactions (checkout vs just browsing the store) and other conditions that can change the affinity of your user (sometimes dynamically.) keepalived isnt mentioned once, or .1q trunk optimization, or SRV records and failover/HA thats performed in most modern browsers based on DNS information itself.
I went down this rabbit hole and was surprised how all over the place the behavior was against various http clients (not just browsers). Very little consistency in how the IPs in the dns response are retried, if at all.
This post is focused primarily on Layer 7 load balancing, connection and request routing based on application-level information, so it doesn’t go into Layer 3/4 techniques like DSR or network-level optimizations. Those are certainly worth covering in a broader series that spans the full stack.
