To be honest, I was using internet as shorthand for average of human knowledge, on the basis that most books, peer reviewed articles, and everything else is already on the internet, even if they exist only in the more unsavory corners (I've seen nothing to suggest the FM producers were / are much bothered about where the data is from).
But yes referring to base models. I'm also not convinced that the average book is any more trustworthy than the average webpage, whether that be a purely technical book, where you really need to webpage of errata to be able to use the examples. Or the more pop-sci books that cherry pick data and jump to completely unfounded conclusions (I'm thinking of the ancient engineers - aliens built the pyramids books).
The feedback is great and might work in some areas, technical knowledge. But once you step outside of the physical sciences and engineering, you don't so much end up with better quality information, just a curated experience that aligns with the model owners (think DeepSeek and Tiananmen square)
