Having a proper (intra-application) messaging system makes a certain class of problems -- problems that you just get used to in Java, and Go, and Rust, and C/C++, and whatever -- just go away.
Interesting. Examples?
1) Data is passed between threads/processes using a message bus (essentially, a queue). This means, no need to synchronize data access, as every subscriber gets it's own message copy.
2) Suddenly, you need access to certain data from another module. If the program was architected as regular class hierarchy, you would somehow need to pass the reference to that required class instance or use a global function call. If there is a message bus where the required class is already publishing messages, you can simply subscribe to data from the required module and it is delivered via message bus.
3) You can replace modules easily. The only interface to other modules is via message bus, so you can replace the module with one that publishes the same type of messages.
4) If the message bus is designed correctly, you can move the modules/process to different machines, and they can access the message bus over network. Bus broker just sends the messages over TCP instead and that is totally transparent for both involved nodes.
Btw, I am still looking for an infrastructure library just like that: https://news.ycombinator.com/item?id=14222202
On the web page it states that Apache Kafka is a distributed streaming platform.
Kafka has four core APIs:
The Producer API allows an application to publish a stream of records to one or more Kafka topics.
The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.