"I always associated semantic web with something Google was pushing to assist with web crawling, and my first exposure to it was during the Web 2.0 era (early 2010s) as HTML5 was seeing adoption, and I always associated it with Google trying to enhance the web as the application platform of the future."
This sounds more like "indexing" than "crawling"
The "Sitemaps 0.84" protocol , e.g., sitemap.xml, was another standard that _was_ introduced by Google
Helpful for crawling and other things
(I convert sitemap.xml to rss; I also use it to download multiple pages in a single TCP connection)
Not every site includes a sitemap, some do not even publish a robots.txt