undefined | Better HN

0 pointswredcoll11mo ago0 comments

I mean... knowing how to solve it isn't the trick, it's doing it a million times a minute for your firehose scraper.

0 comments

Anubis adds a cookie name `within.website-x-cmd-anubis-auth` which can be used by scrapers for not solving it more than once. Just have a fleet of servers whose sole purpose is to extract the cookie after solving the challenges and make sure all of them stay valid. It's not a big deal

fc417fc80211mo ago

Requests are associated with the cookie meaning you can trace and block or rate limit as necessary. The cost of solving the PoW is the cost of establishing a new session. If you get blocked you have to solve again.

j / k navigate · click thread line to collapse

0 pointswredcoll11mo ago0 comments

I mean... knowing how to solve it isn't the trick, it's doing it a million times a minute for your firehose scraper.

0 comments

udev409611mo ago

fc417fc80211mo ago

j / k navigate · click thread line to collapse