undefined | Better HN

0 pointsxauronx12y ago0 comments

Oh very nice. I was doing some server side scraping of HN content a little while back and it was a pain. I would have much preferred to use an API or something. Have you gotten banned a bunch of times yet or are you getting around that somehow?

0 comments

2 comments · 1 top-level

blackjable12y ago· 1 in thread

I'm not hitting it that often and no one is really using the API yet so I haven't hit any limits.

If I do get banned because of excess use I would stop and put some caching in place or something. I didnt want tot start with a cache because I wanted information from the site to be up to date as possible. HN data changes really quickly, as you know.

xauronxOP12y ago

Yeah, absolutely. I got into issues when I was trying to parse the comments for the top 20. If you hit the top 20 and pull down the comments you'll get banned. I ended up giving up on my iOS app because once any real usage started the server was instantly banned (even with 5 minute delay between updating). I ended up realizing that I would have to parse it client side (in obj-c). But then again, I'm not that smart so maybe I was over complicating it.

j / k navigate · click thread line to collapse