Also lets you restart either end of the connection transparently to the other end.
With fd passing, if the daemon I'm talking to dies or restarts my fd is now stale and i have to get another.
Also allows starting things on demand similar to inetd.
Also allows transparent multicast.
So yeah, fd passing would be faster, but routing through the daemon is easier.