Here's my pretty graph for serving ~500,000 requests per minute on Heroku:
https://api.playtomic.com/load.html
Savings aren't exclusively because of Heroku, I also switched the underlying architecture during that migration from C# / ASP.NET to NodeJS which is exceptionally well suited to what I'm doing there.
Previously:
Dedicated servers, ~$1600 a month
8x dedicated servers at ~$200 each, each running 3 to 5 instances of the API depending on how many IP addresses they were provisioned with. Uploading was done via a simple hand-rolled script that'd just FTP everything to each server.
Now:
Heroku, current usage $400 - $500 a month
With Heroku I don't have to worry about concurrent connections (typically 200 - 400 thousand people at once), I don't have to maintain all those servers and I don't have to fuck around with all the stupid things that can go wrong when you're operating at scale.
It was a lot of work to get to this point and I made a lot of mistakes like having a heavy redis pub/sub outside of the EC2 network that cost me $350 in excess bandwidth providing inter-dyno communication, and I saturated database connections lots of times because in the old days those dedicated servers each had a local mongodb which could keep up with ordinary connection pooling, but it was totally worth it.
