undefined | Better HN

0 pointshombre_fatal9mo ago0 comments

TFA isn’t talking about crawling to harvest training data.

It’s talking about Perplexity crawling sites on demand in response to user queries and then complaining that no it’s not fine, hence this thread.

0 comments

cjonas9mo ago

Doesn't perplexity crawl to harvest and index data like a traditional search engine? Or is it all "on demand"?

lukeschlather9mo ago

For the most part I would assume they pay for access to Google or Bing's index. I also assume they don't really train models. So all their "crawling" is on behalf of users.

j / k navigate · click thread line to collapse

0 comments

cjonas9mo ago

Doesn't perplexity crawl to harvest and index data like a traditional search engine? Or is it all "on demand"?

lukeschlather9mo ago

For the most part I would assume they pay for access to Google or Bing's index. I also assume they don't really train models. So all their "crawling" is on behalf of users.

j / k navigate · click thread line to collapse