Do not use Selenium for web scraping (opens in new tab)

(buklijas.info)

5 pointssasa_buklijas7y ago2 comments

2 comments

> The only exception that I could see for using Selenium as web scraping tool is if a website that you are scraping is using JavaScript to get/display data that you need to scrape.

Like 95% of web sites?

sasa_buklijasOP7y ago

Scrapy does have the solution for JavaScript with Splash, but I have never used it, so far I always found some workaround.

j / k navigate · click thread line to collapse

2 comments

snaky7y ago

> The only exception that I could see for using Selenium as web scraping tool is if a website that you are scraping is using JavaScript to get/display data that you need to scrape.

Like 95% of web sites?

sasa_buklijasOP7y ago

Scrapy does have the solution for JavaScript with Splash, but I have never used it, so far I always found some workaround.

j / k navigate · click thread line to collapse