For persistent, high-throughput traffic, Muti Metroo maintains long-lived connections and multiplexes multiple logical streams over a single peer link, each with independent flow control. This works well for token streaming, where low latency matters more than raw bandwidth. In residential networks, QUIC is usually the best choice, with HTTP/2 and WebSocket also available.
Service discovery is handled via the port-forwarding model. A node can advertise a named endpoint (e.g. an Ollama instance), and another node can bind a local listener to that key. The mesh routes traffic end-to-end encrypted, so from the client’s perspective it behaves like a local port even though the service is remote.
For distributed inference, the main constraints are latency and hop count - extra hops add delay, which is fine for background work but relevant for interactive use. Everything runs in userspace, and outbound connections plus QUIC make it usable behind typical residential NATs.
