Whenever I went for low latency, I noticed 2 things:
1. Actual latency was not what I set in the encoder because of frame sizes, chunk sizes, tcp Windows, decided buffering
2. Forcing super low latency caused drop-outs, which is worse for music than delays.
Overall I just found flac to be rock solid and lower latency without much tweak
