Your external DNS server is quicker than a local cache? My local cache adds less than 1 millisecond latency to an uncached lookup, and answers queries for all LAN computers in less than 1 millisecond as well.
My DNS server is pretty fast under ideal circumstances (under 0.07ms per reply using 2000 era hardware as per https://maradns.samiam.org/speed.comparison.html ). I’m sure you’re not getting 1ms in less-than-ideal circumstances (router overloaded and dropping packets, which sometimes happens on my home network), where that extra DNS server starts to really slow things down.
Ya my network never drops packets, at least for congestion reasons. Seems like congestion will affect external servers at least as much as internal ones, though.
(Access to my DNS server is not routed on my LAN, it's a flat network.)