Why would publishers allow you to crawl their sites if you're not sending them any traffic?
The big publishers certainly won't let you do that as they are selling their data to Google, Microsoft, Facebook and whoever else has the money to train a fully fledged LLM, which is certainly not everyone.
