zader

I run a niche dating and social networking website. After I built my own traffic analysis suite, instead of just relying on Google Analytics, I realized a ton of my bandwidth and server resources are devoted to serving up content to bots.

Many are valid bots whose spidering I welcome, but some of the most aggressive do not identify themselves as bots and are from places like North Korea and Russia. So I'm researching solutions, and the best I've come up with so far is using a bad_ips table in my Rails app to block addresses such as the ones listed in the blacklist at myip.ms.

How are other online entrepreneurs dealing with this phenomenon? Are there pre-existing solutions out there that are worth using, or should I proceed with my own custom model? Can any of you recommend best practices or seasoned advice in this area?