undefined | Better HN

0 pointsmrkurt4y ago0 comments

Stale replicas are a "stop serving requests from a particular region" problem. We pretty aggressively monitor these and consider things broken if replication is lagged more than about 10s.

One thing about running thousands of DB clusters (which we did at Compose.com before this) is, these problems exist even without geo replication. And they exist at such a scale that you have to be good at handling them. Geo replicas are _more_ prone to these kinds of issues but we already had to be good at handling them.

0 comments

1 comments · 1 top-level

dkhenry4y ago

That is good and should prevent against bringing down the primary. There is still the issue with being able to get that region back online once replication is severed, which isn't always possible even with co-located systems.

j / k navigate · click thread line to collapse