It's packet-based, and you can delay a packet by a lot of time and it won't cause any problems. So at higher levels there shouldn't be any tight tolerances.
Signal integrity and timings are really important at the physical layer, but I don't think that's what's breaking.