Podcasters get access to the entire data set for every listener and can run clustering and regression to their hearts' content. As you listen to podcasts and perhaps rate them, the cost to target you (i.e. your random ID) goes up because the podcasters have better data to work with and should know better than to send you politics podcasts when all you listen to is vampire erotica.
If you become a listener, then they've acquired a listener and you've acquired a podcast that you like. If you're not interested, then they pay the cost of poor targeting, and you are compensated for the inconvenience. Proper incentives all around!
The net cost to the podcaster doesn't have to be $0 for the service to be worthwhile. It only has to be more effective than AdWords and other venues where they pay to expose themselves.