undefined | Better HN

0 pointsworkhere-io11y ago0 comments

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine?

They can just use PhantomJS (http://phantomjs.org/), which is free and open source.

0 comments

They could, but I wonder what it'd take to scale it to crawling that number of pages.

I think only Bing would have the cash and resources to build that.

j / k navigate · click thread line to collapse

They could, but I wonder what it'd take to scale it to crawling that number of pages.

I think only Bing would have the cash and resources to build that.

j / k navigate · click thread line to collapse