HAProxy 2.0 (opens in new tab)

What open-source NGINX lacks that open-source HAProxy has:

* ACL rules with full support for logical if statements [1]

* active health checks

* end-to-end HTTP/2 [2]

* Robust logging or a dashboard with metrics

* The ability to read env variables

* session stickiness

* DNS service discovery [3]

These are just things I'm aware of, there could be a lot more.

HAProxy has shown itself to perform better for certain users such as Booking.com [4]

[1] https://www.nginx.com/resources/wiki/start/topics/depth/ifis... [2] https://trac.nginx.org/nginx/ticket/923 [3] https://danielparker.me/haproxy/nginx/comparison/nginx-vs-ha... [4] https://events.static.linuxfound.org/sites/events/files/slid...

I'll let others comment on the technical differences.

Have you read The Maglev paper out of Google? At the time right before that was published, one of my engineers was implementing much the same thing. They considered both HAProxy and nginx for the second layer of load balancing. The were technical reasons in either direction. There was some pressure (from me) to go with nginx because we were already using it as our primary webserver, so we could avoid introducing another new technology for people to master.

We went with HAProxy. Why? Because when said engineer contacted them, describing what he was doing, they (here: the main HAProxy dev) engaged in discussion, helped, even included his needs in their planning. At least at the time, nginx folks just responded with that they'd talk to him after he had secured a licensing deal. The uphill battle this engineer would've had to fight in corporate politics to get licensing sorted out that early in the prototyping phase would have been rough. Last I heard, the company still had a licensing/support deal with HAProxy.

Good presales matter!

(Edit: can't spell.)

I’ve never used either (just an F5 appliance), but I know HAProxy is built by a core Linux kernel contributor and has a reputation as the foremost infrastructure-grade software load balancer. In fact, a lot of hardware load balancer appliances run HAProxy under the hood.

From all accounts, if you really need a load balancer, or even if you just need failover, HAProxy should be your default choice. It has high-availability features, monitoring, supports TLS & SNI, HTTP2, session replication, Lua scripting, and almost any other feature you might need. It was also designed from the ground up to be a high-performance, high-availability load balancer. (NGinx does several other things.)

Stack Exchange (the company behind Stack Overflow) uses it in front of their IIS application servers, and so do a lot of other smart people.

Also, it appears NGinx has commercial/OSS conflict of interest issues — like in recent versions all monitoring functionality was removed from the OSS distribution.

In my previous company we used to use HAProxy, and it was a hassle. Yes, it is powerful. However, nginx is way easier to configure and set up, and performance wise is a contender for most usual applications people needed.

Maybe for a few edge cases, HAProxy works better, but overall, I'd pass on it.

nginx just fulfills most people's requirements for reverse proxy and has solid HTTP/2 support (and other features) for way longer.

If you are using nginx and it is working good, I'd recommend against trying out HAProxy.

If it's not working good, I'd first look into fixing whatever is wrong with your setup and only trying HAProxy if some experienced with it helps you out with it. HAProxy requires much more configuration tweaking than nginx (at least to gain any benefit from using it).

For me it's the superior HTTP / header rewriting capabilities. With nginx you are more or less restricted to just adding headers the last time I looked into it.

Disclosure: I'm a community contributor to HAProxy and I help maintain the issue tracker on GitHub.

From what I remember, HAProxy is better at having detailed monitoring metrics, while in the case of nginx a lot of those monitoring features are nginx plus only.

HAProxy allows easier proxying of WebSockets (without having to know all the WebSocket URLs). It also allows proxying TCP traffic without much hassle (in nginx you have to compile/enable stream module - at least in default Ubuntu package). Until long nginx did not allow proxying HTTP requests without caching request body which may be undesired behaviour if you are uploading big files and want the backend to start processing the body as soon as possible.

It's part of our infrastructure at work to do some fairly complex routing, which might be possible in nginx but is easier with haproxy.

Multi-tenant hosting for customers with lots of custom domains (with SSL) with different customers on different versions, and an app with lots of legacy. In older versions, different paths are handled by different servers. Some paths use source-ip sticky. One part of the app uses websockets. Some paths are handled via S3+cache layer (offloading traffic from app servers without changing the app).

There's also a bunch of special paths to access specific servers directly to get some health metrics (from app written without thinking about running in an auto-scaling environment). One fun thing I built handles some old SOAP requests from a defunct service running on hundreds (maybe down to dozens now?) of external systems that will retry every request forever (exponential traffic growth) if they don't get a "success" response; using Haproxy and some request capture regexes, it can return one of a dozen specially crafted hard coded "success" responses. The date is wrong but the service doesn't care, and now few lines in Haproxy replace a dedicated "black hole" app server we used to run.

Haproxy handles all this in a single hop for all traffic (and Haproxy itself runs in an autoscaling group). The config is complex, but still understandable. All the variable stuff is generated by scripts at runtime, which also lets us use an admin UI to manage customer domains, versions, and automatically picks up available (deployed) application versions in the region.

Having used both, I'd say in general nginx is easier for simple things, and in many ways has more capabilities (like static hosting, authentication support). In fact we use nginx on the Haproxy servers for hosting static pages and being a caching proxy for S3. Haproxy makes simple hosting more complicated, but you get a lot of very fine-grinned control over everything (eg, it's pretty easy to do almost anything to the traffic like changing request/response headers/paths at any point). For anything new, I'd use nginx only if I could get away with it (note: all the above may be possible in nginx now, but I'm not going to rewrite our infrastructure unless there's a very good reason).

I've found it easer to do some traffic-shaping with (i.e. queue up more than N connections per ip). The status page/dashboard is really nice. The English docs are better. This might have changed in the last three years

One advantage is that HAProxy has is the built-in status page. In order to get something similar in Nginx, the last time I've tried (a couple of years ago), the only option was a 3rd party, buggy, plugin.

The Problem with HAProxy is largely configuration. I've written ansible templates that generate the complicated and repetitive parts of that config for me.

Other than not logging to stdout (which 2.0 seems to fix), that's the only thing that bugs me about HAproxy.

HAProxy is probably the best proxy server I had to deal with ever. It's performance is exceptional, it does not interfere with L7 data unless you tell it to and it's extremely straightforward to configure reading the manual.

I use HAProxy as a frontend proxy to distribute requests into my intranet, it handles about 3-50 requests per second on a very dinky little VM (1 CPU core, 512/1024 CPU time allocation, 512MB ram) with a lot of CPU still left over.

The biggest difference a few years ago was HA Proxy could do layer 4 load balancing where nginx was layer 7.

These days I think they’ve grown a lot closer together on capabilities. nginx remains a true web server where HA Proxy is not.

That's not usually a choice. nginx is first and foremost a web service and a proxy. It is not built primarily for rewriting traffic and simple run time state changes, which is what you generally need an http router for

Normally you choose between something like Apache and nginx, where nginx is a bit easier to configure and write modules for, but the former has more functionality and third party support.

You choose between haproxy and something like Varnish, where the former is a bit more featureful on the routing part and the latter has more focus on the caching part.

It is not uncommon to use both.

It is very usable to proxy other services than HTTP.

Without knowing the scale it's hard to answer this. But in general HAProxy is THE layer to sit in front of web servers. From DDoS protection rules to a true set of metrics and a real dashboard (good luck w/ Datadog metrics for Nginx), it's purpose IS to be a highly available Load Balancer/Proxy.

Nginx in comparison is for me, THE thing to use as a web server. When it comes to being a single layer away from reverseproxying requests to apps running on an instance or static file serving with caching rules Nginx is purpose built for this.

Both apps have support for what each other do though. For the more pedantic it's easy to say "But x can do this too with this feature". Usability wise though, nginx and HAProxy are distinct. I went through the comparison very recently while testing setup of HAProxy as a load balancer in front of multiple webservers running Nginx which in turn sat in front of several apps running on each instance.

TL;DR - At a slightly larger scale (scale being number of instances + apps) HAProxy and Nginx are great to use together as opposed to one over the other.

Nginx is a web server that happens to have reverse proxy functionality (similar to Apache).

HAProxy is a load balancer with philosophy of "do one thing and do it well".

Perhaps things changed with nginx, but when I tried it for load balancing it was very basic, it didn't even have status page or health checks unless you purchased the commercial version. It only had round robin load balancing method etc.

If your goal is load balancing haproxy is far superior, if your goal is to have a web server that hosts some static files and then redirect dynamic requests to another app then nginx might be better.

haproxys config format is concise and approachable. I like the cbonte single-page html doc page where ctrl+f to elaborate explanations can go a long way clearing up (mis)understandings.

Big difference is that haproxy did not used to support ssl without using something external like stunnel -- nginx basically did it all out of the box and I haven't had a need for haproxy in quite some time now.

Nginx it's more general purpose, ex: can serve and cache static files.

Yes. Any time I need to use cgi's, php, or anything where security outside of Apache can be controlled by an Apache module, I will always default to Apache as the security control story is better. Performance wise, Apache 2.4 using the latest APR libraries is equal to NGinx.

There are also far fewer bugs and updates to the Apache core. I rarely have to recompile anything.

I have also had many frustrating interactions with the lead developer of NGinx. There are many assumptions made and many things hard coded in the Makefile, especially as it pertains to pcre, zlib, openssl and CFLAGS, LDFLAGS, etc. Also, I can't just point to existing pcre and zlib deployments for inclusion. NGinx wants the source and to recompile the extra libraries each time.

Yes, its still more popular than Nginx according to https://trends.builtwith.com/Web-Server/Apache https://trends.builtwith.com/Web-Server/nginx

Of course, for anything that is cgi/fastcgi. nginx doesn't support that.

There is also stuff running on mod_php/mod_python/mod_wcgi that is bound to apache, however these are deprecated and unstable technologies that should not be used in this decade.

What conversation?

Been using apache2 for like 20+ years now. It is doable to switch to something else, but would probably require effort with various details, etc. It works well for our moderate loads, so not really urgent to change it.

You have Lua in haproxy if you want to do complex things. If you just want a programmable routing table, the maps in haproxy do exactly this and can be updated on the fly (including from the traffic itself if needed).

Or you can write the controlling program in the language of your choice and run it outside as an SPOA agent. Have a look at spoa_server which provides examples for Python and Lua.

Shameless plug. I wrote a REST wrapper around HAproxy some years back. Wrote it in Go. Works pretty nice. Willy was a great support also. https://github.com/magneticio/vamp-router

Works quite nicely and can set ACL’s dynamically.

You can do Lua on HAProxy.

Traefik is a very good programmable proxy.

By programmable you mean configurable via a REST api?

maybe openresty? nginx with embedded lua plenty of library support for external systems like redis, memcached, mysql, etc...

As of last April, several implementations (including HAProxy) were more "right" than Nginx: https://twitter.com/tunetheweb/status/988196156697169920

HTTP/2 support was added in 1.8, at the beginning of last year.

AFAIK it's solid.

As in nginx that still doesn't have support for h2 in the backend?

AFAIK nginx does not "do it right", it can get HTTP/2 prioritisation pretty wrong and when it gets it right it can appear to be more by luck than design

https://blog.cloudflare.com/nginx-structural-enhancements-fo...

h2o is probably the only server that has done HTTP/2 right fora long time, others a finally getting it right

Apart from a few new warnings for long-deprecated options it is compatible. HAProxy 2.0 is not a major version. Willy apparently just dislikes two-digit versions in the second place.

Yes, the same configuration should work between both. Although some options are no longer required to be set such as nbproc/nbthread. To be safe pass it through the configuration checker (-c on the command line).

Strange that you see no option for client certs because that has been supported from day one. In addition we even support SNI-based client auth even with wildcard certs. Same for TLS versions and cipher suites.

Further, just look at https://istlsfastyet.com/ and you'll see that haproxy, H2O and nghttpx are the only 3 implementations checking everything (and haproxy was the one inventing dynamic record sizing).

So it seems your opinion on haproxy's TLS support is not that spread!

As you explained, HAProxy does support OCSP stapling through flat file, but also support it through the runtime API.

v1 of the ingress controller does not update OCSP. That said, this is planed for a next release.

Stay tuned :)

It definitely depends on your use case, so it's hard to tell what's better for you. HAProxy is solid and doesn't take a long time to get started.

At the same time, some of the HAProxy 2.0 features have already been available in Envoy and tested in production, at scale (if HAProxy provided those features, there wouldn't be a big need for Envoy). For example, Envoy is pretty extensible, has good performance and has good support for dynamic cert management (including service-to-service mutual TLS).

Envoy was built for that purpose and has more functionality around it, as well as better support for service discovery (developing the now open standard APIs), more protocol introspection and observability, full-duplex connections (no upstream/downstream split in what's possible), and easy interchange between protocols.

Envoy is also used by Istio and has a lot of infrastructure support for deploying in Kubernetes and such which HAProxy doesn't currently have.

If you're seeing good support for musl, I'd be interested in receiving a patch to add it as another combination. I prefer to keep the libc apart from the kernel (the mistake we made long ago was to mix them) so that we don't have issues anymore when building on other libcs. For example getaddrinfo tends to be bogus on uClibc and must not be enabled there. And threads do not work on dietlibc if I remember well.

Same here, except for a Consul integration, so we didn't have to rely on SRV-records, but I guess you can't have everything :)

I would check out https://github.com/caprover/caprover. You can run multiple apps on 1 VPS and HTTPS renewal is automatic.

You can also try Caddy: https://caddyserver.com/

To be fair to squid and nginx, they don't do the same things. Squid is mainly a forward proxy. Nginx is mainly a web server. There's no reason for not using them anymore for these use cases where they excel.

Although HAProxy is not a web server, it does have Small Object Caching so files can be cached on the proxy. https://www.haproxy.com/blog/whats-new-haproxy-1-8/#http-sma...

It can proxy and cache them.

There is a hacky way to serve single page static files directly via haproxy, by creating a http file (including headers) and add it to a custom back-end. This is really only useful if you have one or two pages you need to serve and don't want to run another web server. Some people use this for outage or maintenance pages.

    acl landing path_beg /demo
    use_backend landing if landing

    backend landing
        mode http
        fullconn        10000
        errorfile 503 /opt/ha/landing_page.http
        http-request    set-log-level silent


    # cat landing_page.http

    HTTP/1.1 200 Ok
    Cache-Control: no-cache, no-store
    Connection: close
    Content-Type: text/plain

    This is a static text file served directly from HAproxy 1.9

    I will catch grief for showing this.

Then use acl's to send specific requests to that page. Obviously change/add headers as required for the content in the file and use-case. Perhaps useful for outage or maintenance pages. Use your imagination for other use cases where you might want to automatically serve such a page.

Here it is in use: [1]

[1] - https://tinyvpn.org/demo

This is a strange assertion. This is not envoy, it's haproxy as you've always known it plus all the features people have been asking for recently, without removing what makes it fast, robust, compact and flexible. From what I've seen you can't for example use dynamic weights in envoy, protect from DDoS, perform queuing to protect your servers, use true leastconn or weighted hash/roundrobin, stick on arbitrary information nor synchronize it between members of the cluster, create complex routing rules, set the source address from headers, perform transparent proxing, etc.

These are two different projects. One was initially designed for the hostile edge and excels here. The other one was initially designed to be used as a side car deep into your infrastructure and excels there. There is obviously quite some overlap between the two, sometimes with different terminology (like "circuit breaking" in envoy that haproxy calls "timeouts" and "queue limits"), and users demands make each of them evolve a bit in the area they are less good (i.e. where the other one excels). But they are still quite different beasts.