undefined | Better HN

0 pointssam0x172y ago0 comments

Furthermore, all bots worth their salt as far as threats go enable js and do everything they can to appear like a normal browser.

0 comments

4 comments · 2 top-level

azalemeth2y ago· 2 in thread

I'd love to know if puppeteer passed that test (probably). I have had exactly this problem many, many times and it is incredibly frustrating.

sam0x17OP2y ago

There are github projects that are forks of things like selenium and puppeteer that are specifically designed to avoid detection for things like scraping google search results, etc.

20after42y ago

puppeteer passes the test if you run it from a machine that already has a good cloudflare reputation score. Try it from an AWS instance and it definitely fails 100% of the time.

(I've tried it, that's how I know)

tick_tock_tick2y ago

That's fine as processing the javascript increases the coast at-least.

j / k navigate · click thread line to collapse