Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
0 points
spiderfarmer
2mo ago
0 comments
Save
Share
They work because they offer ways to opt out, they honor crawl delay, setting ideal scraping times, IndexNow, etc.
And they give you real, valuable traffic in return.
0 comments
1 comments · 1 top-level
top
newest
oldest
avazhi
2mo ago
Most offer ways to opt out, some don’t. Scraping somebody’s website might be annoying or problematic traffic-wise but that’s a far (very far) step removed from saying scrapers should be criminalised. The latter statement is outright laughable.
j
/
k
navigate · click thread line to collapse