> Crawling and scraping is legal. If your web server serves the content without authentication, it's legal to receive it, even if it's an automated process.
> If you want to gatekeep your content, use authentication.
Are there no limits on what you use the content for? I can start my own search engine that just scrapes Google results?
There are many APIs that scrape Google but I don't know of any search engine that scrapes and rebrands Google results. Kagi.com pays Google for search results. Either Kagi has a better deal than SERP apis (I doubt) or this is not legal.
I tried to scrape Google results once using an automated process, and quickly got banned from all of Google. They banned my IP address completely. It kind of really sucked for a while, until my ISP assigned a new IP address. Funny enough, this was about 15 years ago and I was exploring developing something very similar to what LLMs are today.
I think OP based this on an old case about what you can do with data from Facebook vs LinkedIn based on if you need to be logged in to get it. Not relevant when you talk about scraping in this case I think. P is clearly in the wrong here.
