The most important thing that I found out while doing my research is it's not the fastest bytes that win. Well, that does matter, but it's not that important. It's reducing performance variance [1]. While my project wasn't optimizing for speed, it was optimizing for zero-copy de/serialization, but this often ends up as a solution for high-speed transfers. SBE, Flattbuffers, Cap'n Proto all had their places, but I ended up not using any of those and just hand-rolling something similar to what SBE would do. If this was a $DAYJOB project I'd probably end up doing something with SBE.
[1]: https://speice.io/2019/07/high-performance-systems.html
TCP has no concept of a message, you need to build it on top of TCP.
Which HTTP does, but it doesn't have a standard concept of a message format, you need to build it top of HTTP.
Which gRPC does, but it closes the connection after a successful exchange.
Which is avoided by streaming gRPC - makes sense if you know you'll be talking more over this channel.
None of those protocols are really comparable, and most of the differences boil down to the serialisation protocol (binary/proto or textual, like JSON).
- Suddenly you want to enable TLS between one or more components, meaning you need to wrap the socket inside a TLS channel.
- You discover that your client behaves poorly when servers go offline, so you add your own logic for keepalives/pings.
- At some point you want to add metrics to all of this, so you decide to manually add Prometheus metrics to the client/server.
- Later on you want to attach OAuth2 tokens to requests as well, so that you can do credential passing.
- In order to get more insight in your setup, you decide that you want to use this in combination with OpenTracing/Jaeger.
Once all of those features are added to your Redis-like protocol, you discover that you've basically reinvented gRPC... poorly.
For fullstack dev, I've been immensely happy with grpc/protobuf over http because of the type safety I get communicating between Golang and Typescript. This eliminates a whole class of bugs but is only a serialization benefit.
Generally: use the thing with the best tooling!
I want to write my integration test suite for the back-end service in typescript to hopefully be able to reuse some of the test suite code for the front-end. But I'm struggling to set up a functional CI pipeline.
Drop me a line at parham@cloudsynth.com and happy to give guidance where I can.
To repeat, grpc has no "type safety" whatsoever.
Happy to consider using that term instead. For most folks, the one I chose is good enough.
While GRPC has its place it also comes with headaches like pretty lousy generated interfaces, horrible debuggability, and unpredictable scaling.
1. Having a complex build system that is aware of gRPC
2. Massive migraines
Unfortunately build systems integrations, IDE integrations, and generated code is unanimously awful for gRPC.
Every company I've seen use grpc has unfortunately adopted the practice of basically manually running protoc and committing the generated code into repo - sometimes modifying the code manually to make it import successfully (Python).
I hope Bazel evolves into a state where it's usable by the average engineer and has first class support for gRPC.
Section 3.5 "Moving code styles" (and 3.4 for RPC):
https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch...
Last time I used ZeroMQ was ~7 months ago at my previous job, so maybe things have changed since then, but having to write hacks to see if the other side is still there or not is absolutely terrible.
Also, the Python bindings did not like fork() without exec, which yes is bad and all, but is still something that is done every so often in real world large programs.
I believe GRPC is pluggable so if one wanted to invest time in building a GRPC ZeroMQ Transport then that is a feasible route. GRPC bring really good RPC mechanisms to the table.
Atom is an easy, Redis Streams-based RPC that also emphasizes docker containerization of microservices. We support plug-ang-play serialization with msgpack and Apache Arrow currently supported and more on the roadmap. You can also send raw binary if you please.
Another nice thing about Redis is that if you're running microservices on the same instance you can connect to redis through a linux socket on tmpfs and bypass the TCP stack to get even better performance.
Anyway the problem with comparing rpc protocols is that you need to do a per case benchmark if you really care about performance. Pretty much every decent solution will be better than another equally good solution depending on the use case.
Years ago, one of my customers told me he used to work doing high frequency trading in a bank, and the bank had several tailor made solutions for data serialization and RPC made for very specific cases, and they were just better than any generic solution.
The main benefit was we could suddenly reuse all the codegenerated routers/docs/authentication from the HTTP ecosystem. It significantly simplified/standardised our IPC layer and reduced the "weirdness" in the codebase.
The overhead of Http (especially 2/3) matters so little on modern hardware.
* AMQP 1.0 - can also be used for RPC without a broker in between client and server. See https://qpid.apache.org/proton/
* Aeron - low latency, UDP based, see https://github.com/real-logic/aeron
Example “blocking” client https://github.com/EnMasseProject/enmasse/blob/master/amqp-u... , but might give an idea of how to set “dynamic source” required for rpc.
In general though I think the Qpid python and c++ examples might be better.
This is just as bad as front-end web dev!
If the channel is marked as persisting, I put the message to a hashset with current nanosecond as the timestamp and just send a ping instead.
It is pretty fast with asyncio redis client clocking at an avg 15k rpc/s on a single core with uvloop on a lowly i7
RPC over Rabbit is fine if you don't care about the result, or you can guarantee that each message gets processed in a short constant time.
1) Don't requeue on error. One bad message could bring down your entire service. Better to just push it to Sentry and make a fix for it.
2) Have a timeout in your message handler.
I would not consider using it for production code either.
