Today I gave it another shot, and after half an hour of trying to glean technical detail and skipping any parts that read like a like a comic book ("zap-pow-kaboom satori paradigm-shift moment"), I think I've learned the following:
* ZMQ works by spawning a background thread that runs a
poll() (or equivalent)-based async I/O loop. This thread
is created when you create a ZMQ "context."
* On top of a context you can create a ZMQ "socket", which
may map to N underlying UNIX sockets (or a comparable
transport), and which will transparently queue data to slow
or unavailable receivers.
* Over a socket you can send or receive "messages", which
are length-delimited strings which are always delivered
in full with the original length. Send/receive can be
either blocking or non-blocking.
* Contexts can be shared across threads, but sockets are
not thread-safe. Communication between application threads
and the ZMQ I/O thread is via a lock-free queue.
I also think it's kind of a misfeature, and that in the real world it tends to devolve to "hub and spoke client/server with lots more failure modes".
I read the testimonials, which got me excited, so I then tried to make use of zmq in a system I was working on. Using plain old sockets, I could whip up a connection in a few lines (OK, python helps here), and deal with all failure modes associated with them. Mapping this robust setup to zmq, required multiple layers, async requests, and a polling mechanism. I don't recall now, but I may have had to throw a router in there as well. I'm not dissing zmq, as it's a great technology, and solves some complex problems well, but it's not for everyone, and people without complex problems will more likely be confused by it than helped.
The documentation is OK, but I think tries to abstract too much of what's going on, and just leaves you with a bunch of less-than-descriptive names for things that you have to take for granted. All the mamas, papas, and dealers analogies didn't help me at all.
The part that I like about ZeroMQ is that it's a simplifying abstraction, e.g. it's a fairly simple wrapper about sockets that makes life easier for me. I guess it's mostly like a library of patterns for socket usage, so it makes a request-response loop and pub-sub scenarios really easy to implement. The other thing that's great is that it's pretty much programming language agnostic, meaning that C++ code and Python code look fairly similar and can trivially be used together.
As for the transparent queueing, there are simple options to control the size of the queue; it's even possible to spool the queue to disk as soon as the memory limit is reached (see setsockopt docs and look for HWM, for high water mark).
Sorry for the comic book style. It's how my mind works. Feel free to send a patch. :)
The benefits of 0MQ to me, as a programmer, are:
* Really easy to write utterly solid multithreading code, in a language (C) that has zero support for this; * Really easy to take this same code and make it multiprocess, or multibox, with few changes. * Built-in handling of asynchronous I/O, which I need in any real application. * Speed, meaning I can be lazy and my code still runs very quicky. This is less obvious than it seems. 0MQ's critical path is extremely evil.
It's so hard to explain this with a diagram or a slideshow. The only way to understand is to use it, to write code with 0MQ. Certainly for me, when I started programming 0MQ apps, my understanding of what this library was about, and what it could do for me, totally changed.