echo "--no-ri --no-rdoc" >> ~/.gemrc
install every gem much faster, avoid rdoc vulnerabilities. Since gem installation can run arbitrary code, I am suspicious of rdoc xss vulnerabilities being a cause for concern (if you are not running a publicly-accessible gem documentation site...)
