undefined | Better HN

0 pointsmwcampbell5y ago0 comments

> Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config.

What if you used one of the managed RDBMS services offered by the big cloud providers? BTW, if you don't mind sharing, where are you hosting the control plane?

0 comments

bradfitz5y ago

> What if you used one of the managed RDBMS services offered by the big cloud providers?

We could (and likely would, despite the costs) but that doesn't address our testing requirements.

The control plane is on AWS.

We use 4 or 5 different cloud providers (Tailscale makes that much easier) but the most important bit is on AWS.

nicoburns5y ago

Why is testing Postgres/MySQL difficult? You can easily run a server locally (or on CI) and create new databases for test runs, etc.

bradfitz5y ago

It's not difficult. We've done it before and have code for it. See the article.

ignoramous5y ago

> ...but the most important bit is on AWS.

Curious: Was running DynamoDB with DAX (DynamoDB Accelerator) in front ever in contention? If not, is it due to vendor lock-in (for example, not being able to migrate out) or because tailscale doesn't feel the need to use managed offerings especially for core infrastructure?

cakoose5y ago

> > Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config.

> What if you used one of the managed RDBMS services offered by the big cloud providers?

Yeah, AWS RDS "multi-AZ" does a good job of taking care of HA for you. (Google Cloud SQL's HA setup is extremely similar.) But you still get 1-2 minutes of full unavailability when hardware fails.

I haven't operated etcd in production myself, but I assume it does better because it's designed specifically for HA. You can't even run less than three nodes. (The etcd docs talk about election timeouts on the order of 1s, which is encouraging.)

For many use cases, 1-2 minutes of downtime is tolerable. But I can imagine situations where availability is paramount and you're willing to give up scale/performance/features to gain another 9.

j / k navigate · click thread line to collapse

0 comments

bradfitz5y ago

> What if you used one of the managed RDBMS services offered by the big cloud providers?

We could (and likely would, despite the costs) but that doesn't address our testing requirements.

The control plane is on AWS.

We use 4 or 5 different cloud providers (Tailscale makes that much easier) but the most important bit is on AWS.

nicoburns5y ago

Why is testing Postgres/MySQL difficult? You can easily run a server locally (or on CI) and create new databases for test runs, etc.

bradfitz5y ago

It's not difficult. We've done it before and have code for it. See the article.

ignoramous5y ago

> ...but the most important bit is on AWS.

cakoose5y ago

> > Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config.

> What if you used one of the managed RDBMS services offered by the big cloud providers?

Yeah, AWS RDS "multi-AZ" does a good job of taking care of HA for you. (Google Cloud SQL's HA setup is extremely similar.) But you still get 1-2 minutes of full unavailability when hardware fails.

For many use cases, 1-2 minutes of downtime is tolerable. But I can imagine situations where availability is paramount and you're willing to give up scale/performance/features to gain another 9.

j / k navigate · click thread line to collapse