Amusing, but completely hopeless and misguided. This is showing up years late.
Nobody training an AI does so continuously and blindly. AI isn't Google, it's not automatically ingesting the news of the last 5 minutes. An H100 costs I believe $25K each. Nobody spending hundreds of thousands of dollars or millions on training an LLM is just going to throw random garbage at it and hope for the best.
At this point in time a whole bunch of companies are seeing training data as the "new oil" and as a result building a business model based on selling high quality, curated data. This can be at least a starting point for anyone looking for good content.
Also, when we're talking about scouring the Internet for content there's many metrics available. We can see whether those fake websites are actually used by anyone and referenced anywhere. You can use a LLM to try to work out whether they're serious or a joke. You can see content spread from the source to end users.
In reality, such an approach has an extremely minimal chance of working.
