Snappy Dashboards with Redis (opens in new tab)

Just like this also has nothing intrinsically to do with Redis and could be any regular ole key-value store.

I don't know how well Mongo's map-reduce works, but in Postgres, COUNT(star) [1] does not perform well for very large tables (e.g. 100 million rows). You wouldn't want to be doing a COUNT(star) once per minute for each customer that had their dashboard open on a plasma screen.

Of course, there are other solutions to that problem: generate the counts on some more feasible schedule and cache them; have a read replica used for analytics queries; shard by customer and have no large customers.

I don't know whether their scale strictly requires the Redis solution, but in any case there are situations where it's not as simple as "throw it in Postgres and use an aggregate function".

[1] "star" instead of an asterisk to avoid HN thinking I'm trying to write in italics.

Make sense?

Introducing new services/technologies just complicates things on both the application and ops layer (even though the author did say he was using RedisToGo).

Our API is a Rails project and though we do a lot of work that's "off the rails," so to speak, we do try to follow a convention where possible. A convention that's worked well for us is to use Postgres only for database-backed models in the app. So when it came time to solve this problem, Redis seemed like a good choice.

Map/reduce isn't an option here. First, it's too slow. Second, it's asynchronous by nature, which doesn't work well when trying to load a webpage. We did consider using it in the background, however, to generate the counter-caches.

Thanks for reading!

1) They're so easy to move out to something else

2) You main DB probably runs on very expensive hardware, unneccessary expensive for things like this. This probably does not need that expensive san backing it. Not all data is created equal.

3) Redis does this faster

4) Redis is simpler

Of course there are some advantages to storing everything in the same DB, but to me, this seems like a good example of "use the right tool for the job". But, I haven't actually tried this in Redis, so what do I know..

Speaking for ourselves, we've historically preferred to have someone else manage our infrastructure. Setting up Redis is fairly trivial, I agree. However, setting up a secure machine out on the internet and making sure it's always there is less trivial. Rather than juggle an ops/engineer role, we just double down on engineering. I'm sure at some point this may need to change. Hopefully, when that day comes we can afford a sysadmin!

On a related note, it feels good to know that everything our app needs to run is in the code base. Back in my C# days, I remember relying on stored procedures that were configured manually at the database level. Rails fights that with migrations and the callback chain too, so I guess that thinking has sunk in for me.

Thanks for the comment.