Ask HN: legally can i build search niche web search engine?

6 pointsumenline13y ago2 comments

i want to build search engine that searching dally niche sites , like blogs first page's then present statistics based on the crawling. legally can i do this ?

2 comments

yamural13y ago

There is no legal restriction for crawling any web site. But you should respect robot.txt document for every single web site. Such as: https://www.facebook.com/robots.txt http://web.archive.org/robots.txt

manidoraisamy13y ago

Yes. You can. Google successfully defended it in the past : http://en.wikipedia.org/wiki/Field_v._Google

j / k navigate · click thread line to collapse

Ask HN: legally can i build search niche web search engine?

6 pointsumenline13y ago2 comments

i want to build search engine that searching dally niche sites , like blogs first page's then present statistics based on the crawling. legally can i do this ?

2 comments

yamural13y ago

manidoraisamy13y ago

Yes. You can. Google successfully defended it in the past : http://en.wikipedia.org/wiki/Field_v._Google

j / k navigate · click thread line to collapse