digest

I'm curious to hear how you would structure the architecture for an email marketing/CRM service for scale. Let's say you have 5,000 customers. Some customers have 5,000 or less contacts, some have 100,000+, some have millions.

Are you storing all their data on one large database? Multiple databases? If multiple databases is there a specific way you're splitting up the data across these databases?

If multiple databases how are you ensuring search in your app is extremely fast to return any datapoint a customer is looking for on a customer record?

When it comes to email sending, how are you handling it? Lets say you have 100 customers at once send an email blast that consists of 30MM emails. What queue service are you using? What about analytics to track all of the data? Are you storing that data on separate boxes?

What about replication? Set it up on all boxes?

Curious to hear how people would handle a setup like this that will remain extremely fast, all data is quickly searchable from the UI, application is always stable and does not slow down under heavy write operations, and if there is downtime maybe the entire app is not offline but only a few databases that contain only some customers data.