But this is not just nginx's fault. It's also Bad Design on the part of AWS because switching IP addresses this way means you can't keep a TCP socket open to an ELB machine for more than 60 seconds at a time because you never know when the routing rug is going to be yanked out from under you. This makes ELB useless for anything involving a persistent connection. No websockets for you!
If the ELB does get a new address, then yes, the connection will fail, the client will have to reconnect, and when it does, it will need to do a fresh address lookup. But since the connection is over a network, failure is a possibility regardless of what AWS does, and so clients need to be able to detect failure and reconnect anyway.
It doesn't necessarily mean you can't use websockets through an ELB, it just means that you would need to be able to handle reconnects, but that shouldn't be a new challenge for any system relying on connections being open for long. Also, the load balancer servers doesn't switch every 60 seconds, you can have connections running for a lot longer than that. I would also assume the load balancers keep handling connections for a while after they were taken out of the DNS rotation, in order to make sure DNS caches are updated before the IP addresses stops working.
When the "routing rug" is pulled out from under you all you need to do is re-resolve and re-establish the TCP connection which will likely live on for days (in most cases weeks) without disconnecting again.
This is fine for most use cases I am aware of.
As for Websockets, you will need to run the ELB in TCP mode to do that and probably run a real HTTP proxy behind it that supports Websockets/UPGRADE and uses constant source-ip hashing and supports the TCP PROXY protocol. i.e HAProxy. You can run HAProxy or other any other proxy that matches the above in an ELB to get good highly available Websockets proxy layer.
It makes me think that they must be intentionally omitting features in order to make plus valuable. That seems rather cheesy. Obviously it's their code and right, but I think it shows how hard it is to profit off an open source program.
Ideally I would have like it to be an option though, instead of it basically having become a feature in Nginx Plus - if it wasn't for the way I described in the post.
Many applications are programmed this way but I firmly believe that an application needs to honor the TTL of a DNS request for any subsequent connections, for exactly this reason. DNS records change. Sometimes frequently. IMO you shouldn't need yo kick your apps to get them to use the new hostname for subsequent connections.
What I settled on was the following:
Ngninx for SSL termination in front of an haproxy load balancer. I wrote a simple Python script (92 lines of code) that spits out a new haproxy configuration and gracefully reloads haproxy whenever the DNS changes. That has (after fixing a bug or two in my code) been FAR more reliable than any other solution to this problem we've tried.
Bonus: haproxy is a superior load balancer with better runtime metrics and health checking options than nginx.
Also, the author doesn't mention it, but you can also set a custom cache timeout with the valid=Xs option.
http://nginx.org/en/docs/http/ngx_http_core_module.html#reso...
Oddly enough I'd never really considered nginx for the job despite using it to reverse proxy elsewhere. Sometimes you just need a poke in the right direction :)
Now I just need to figure a way to specify *.internaldomain so that nginx resolves www.example.com.internaldomain - where www.example.com is grabbed from the requested Host: header
See: http://nginx.org/en/docs/http/ngx_http_core_module.html#var_... And: http://stackoverflow.com/a/15414811
Always be careful with user input though.
Of course rigorous testing before going live, but it gives options. I love options :)
I know you can use a map in Nginx to do what you ask for, as long as you have a list of domains already: http://nginx.org/en/docs/http/ngx_http_map_module.html#map. I can only imagine it also being possible to make fully dynamic, I just don't have a clear way of doing it in mind right now.
sadd backends:hnapi.dev 192.168.0.42:10555I'll have to look into it further :)
For some reason it's the most popular reverse proxy for this sort of stuff, however it's not particularly well suited for it. We have this issue. We also have the lack of active upstream checks and any sort of upstream status reports. Both are hidden behind the nginx Plus paywall which is absurdly expensive in a micro-architecture world. There are various patches, Patches!?, you can apply but the documentation isn't fantastic and this is an extra bit of hassle we don't really deserve..
I think the open internet deserves a better reverse proxy TBH.
* They were not purposely holding back features so they can be pay-walled
* Re-compiling were not necessary to add functionality
* More cohesive modification ecosystem
I know I said "reverse proxy" which HAProxy is cleary the more superior, but I didn't mean to limit my statement to reverse proxies. nginx does have a lot of other functionality people rely on..
https://github.com/bninja/rump
These are what the routing rules look like:
https://github.com/bninja/rump/blob/master/test/fixtures/set...