Most routers just have a DNS cache, not a global TTL, sorry about the misunderstanding.
I have it set up to be 1 day. Most websites have been setting up low TTLs like 1 hour or less because they don't want to take a chance in case of a problem, and DNS is cheap or free these days.
About your statement - 'don't trackers give out IP addresses?', what do you mean?
Often trackers are addressed by their domain (I've never actually seen a tracker without one). So each torrent has its own specific set of trackers. So all those trackers are frequently resolved and contacted for peer exchange.