undefined | Better HN

0 pointslolc2mo ago0 comments

This, to me, is the strongest argument to offer these slop generators. It provides an incentive to follow the robots.txt.

0 comments

1 comments · 1 top-level

xyzal2mo ago

Exactly. You disobey robots file => we'll make your crawl gain a net negative.

j / k navigate · click thread line to collapse