Just as a point of information, that is not entirely true. Podcasts are tracking you too, but the technology is older and can't do it as well as Facebook.
When you listen to an episode of a podcast, an advertiser can get data from the file download, so at the very least they know your location and your phone profile. Say you download a podcast from a certain location Brooklyn, using iPhone X, an advertiser can profile you to be upper-middle-class and show you a Peloton ad, while same podcast download of an old Android phone is going to get an ad for a short-term loan.
The advertisers are hungry for ways to get more data because right now it's still a black box, you throw money in, and maybe someone buys. They would much rather target with precision, than with banner ads.