undefined | Better HN

0 pointsErikAugust6y ago0 comments

Just crawl, and then disregard sites that have certain scripts present.

0 comments

1 comments · 1 top-level

Yeah, technically, it'd be very easy. I've written countless spiders, crawlers, etc.

Basically, only index items where webpage.indexOf('google') < 0

j / k navigate · click thread line to collapse