In the thing that we're trying nginx terminates TCP (and SSL if needed), proxies back to a unix socket where HAProxy is listening. This gives us the best of both worlds: we can reload HAProxy all day long and socket listeners are hitless, and we rarely need to reload the nginx config which comes out of the box with hitless TCP sockets. But it's not quite as cool as this solution because HAProxy and nginx both have to accept.
We haven't decided that we're going to do our internal load balancing this way yet because the listening ports change so often, but if we do I'll probably be working on https://github.com/airbnb/synapse/pull/203 to do it. The PR makes it so that Synapse can manage both nginx and HAProxy simultaneously (to deal with the extra system complexity of an additional proxy). Or we'll try this cool thing out :-P
The tldr is that in Linux right now there is no way to gracefully drain connections from a listening socket. Many programs work around this by passing the socket file descriptor from the old process to the new process, which is, for example, how nginx works.
