If something in my house was continuously transmitting a stream of audio, I'd notice. Very, very eventually, but I would notice. There's enough of us out there that this sort of thing is harder to sneak by than you might first guess. Home networks are easy to sniff because they're so empty, whereas my work network is a constant stream of mDNS, DHCP, and all sorts of other broadcast traffic to step through before I can see anything interesting.
(Also, yes, I'm eliding details like wired vs. wireless sniffing, etc. And I'm not talking about the router, though evidence online suggests there's a set of people periodically sniffing the router<->internet, too. And yes, clever clogs could try to time things to when people may not be looking, etc. The point is that the traffic is not as unwatched as you may think, not that the watchers are perfect.)