Ask HN: How to load-balance two 1 Gbps connections with a failover?

Edit:

>If 2 Gbps is not possible within a reasonable budget, I can settle on 1 Gbps, that's plenty for me, too.

Even the very old End-of-life PC Engines APU1D (AMD T40E Bobcat) can handle 1 gbps (I have 4). If something new cannot I would look elsewhere.

One way I've shared Internet with two providers is by having two routing machines, each configured with its own Internet connection, with different local IPs on the same LAN. Clients can each be configured to use the preferred gateway, or they can be switched using DHCP. Likewise, when one of the lines is down, the default route can be removed from the machine that handles that line and can be switched to the other machine's internal IP, which will DTRT and will cause clients to use the other. This has the advantage of not requiring waiting for an updated DHCP lease, so switching can be nearly instantaneous.

This is useful if one line is more robust than the other but isn't as fast - the machines that need a reliable connection always use the robust connection, and everything else opportunistically uses whatever's available.

Another is simple round-robin of NAT to both connections.

Neither helps if the line that you're using drops mid-call, but there are ways to deal with that, if you have high speed available at a datacenter. For instance, I've set up routing of a small public subnet via tinc (https://www.tinc-vpn.org) over two routing machines, each with their own Internet connection, with CARP so that packets can go through either machine. This makes handing off from one to the other transparent so that connections don't need to be reestablished.

To get back to your original query, I haven't seen any off-the-shelf NAT router that either does what a host-based router can do without some parts being proprietary or without a good number of drawbacks and limitations, nor have I seen hardware that can do anything fancy (that is, anything beyond the most simple routing / NAT) at high speeds without spending lots of $.

Recent connections that are 2 Gbps and faster are served very well by Ryzen 5600X systems with 2.5 and 10 gigabit ethernet, and those systems cost around $500 each.

Most of my machines have been set up many years ago, and automatic scripts update things like BIND and other software, so they require almost no maintenance after initial setup. On the other hand, remote administration is dead easy because they all use ssh with keys (no passwords), and can be used to help facilitate remote administration of machines on the local network, too.

In other words, I can't think of any reason any more to buy off-the-shelf NAT routers. Even if I wanted to go that route, there are too many shortcomings for me to imagine doing that - I'd give up significantly more flexibility than the amount of time I'd save would ever possibly balance.

To answer your questions about layering NAT routers, yes, you can do that, although it's discouraged. If you're forced to use the ISP's routers, you should at least ask if they can be put in bridge mode so your device can do the NAT, since many of those ISP provided routers have tiny NAT state tables and/or time out NAT states for no good reason.

Also, there's no such thing as NAT that wouldn't fit the definition of "stateful firewall", so it's hard to know what they're advertising. For instance, the Peplink Balance 580 advertises 1.5 Gbps throughput, and that's in aggregate - it definitely can't handle all five WAN at 1 Gbps at the same time, else they'd advertise 5 Gbps. Considering the prices of the hardware, a brand new, physically tiny, host-based NAT router / firewall would be both cheaper and significantly faster.

Just some thoughts :)

Yes. The multi-WAN routers can (or should) be able to manage any sort of WAN over Ethernet handoff; bridged PPPoE, DHCP on public address, DHCP on private address, static IPs. One of those fits what your ISP router provides to LAN clients.

Personally, I'd take the time and build an overly complicated custom system (and I have!), but off the shelf should work too. The tricky thing if you build it yourself is determining when the links are up; for my system, I have PPPoE as primary and low quota LTE as secondary, so it's easier, if the PPP link is connected and has responded to ping within the last 30 seconds, I use that, if not I use LTE. In your case, you might need to build some other liveness check.

Don't expect perfect balancing either --- sometimes multiple bulk downloads will end up using the same connection and you won't have any using the other one, but statistically things should work out.

Otherwise building a pfsense/opnsense box would probably be the route you want to take.