undefined | Better HN

0 pointsbitlad3d ago0 comments

PostgreSQL is good enough until it's not good enough, when you realize all the bad design decisions that were made before it hits scale. It is the decisions people make around not partitioning, HA, replication that makes it not good enough.

0 comments

15 comments · 6 top-level

throwawayffffas3d ago· 2 in thread

These are not bad decisions they are reasonable tradeoffs at the beginning.

Requiring HA, partitioning, and replication are good problems to have.

The alternative is spending engineering time on setting all these up for a failed service with like 100 users.

esafak3d ago

It is not good to have a problem that requires a year of migration by a dedicated team (typical case if you have this 'good problem') when you could have simply spent another day or two to architect it right. You should at least have a path to scale when the time comes.

throwawayffffas2d ago

Deploying HA, partitioning, and replication requires much more than a day or two especially from a team that has not deployed such a system before. And then it requires constant monitoring.

throwaway77833d ago· 2 in thread

Re: bad design decisions - This can be said about any technology.

whh3d ago

+1 - when to prematurely optimise, when not to.

jamesfinlayson3d ago

Oh yeah - I remember working with a guy who absolutely insisted that we build application-level sharding into what we were working on (i.e. each primary key would have some metadata that our application logic would use to route traffic to the correct Postgres instance). Our database was not big - 10s of gigabytes - but Instagram did something similar and we needed to be able to scale. I pushed back hard, and fast forward two years and the application was decommissioned without hitting any sort of Postgres bottleneck.

stuaxo3d ago· 2 in thread

That's fine. There are plenty of projects that don't hit that scale.

thayne3d ago

There are also plenty of projects that do. PostgreSQL is good enough for most projects. But it isn't enough for every project.

jghn3d ago

The problem is that way too many people assume they're in the latter group, when the proportions are very much the other direction

cyberax3d ago· 2 in thread

HA is now pretty good on Postgres.

As for scale... Just use a larger machine. This works for regular transactional data until you're at something like Amazon scale.

Edit:

Think about this, suppose that you store 1 megabyte of data for each of your customers. So if you have a million customers, it's just 1Tb. And these days, you can have a server with 10Tb RAM delivered overnight. Although you might have to sell your firstborn son (offer applies only to royal families) to fund it.

A lot of sharding/no-sql/... development happened in the late 2000-s when computers were about ~100 times less powerful than now. You _could_ get a system with 10Tb RAM in 2010, but as a specially-designed supercomputer.

leoqa3d ago

128Gb of RAM is like 1.5k

cyberax3d ago

For a non-ECC version. A single-module 128Gb ECC stick is about $4500. HBM is even more expensive.

anitil3d ago· 1 in thread

I feel like any problem that Postgres can't handle is a good problem to have. Either you've got so many customers that you're hitting sharp edges, or you're working on such an interesting problem that you're out of the domain where Postgres is helpful. That I should be so lucky

nine_k3d ago

A moderate-size queue is already a problem that Postgres struggles to handle %) But fortunately such problems are rare, and relatively well-known, like, say, doubly-linked lists in Rust.

For most other problems, Postgres works well, or at least well enough.

1 more reply

shepherdjerred3d ago

Everything is "good enough until it's not good enough". That's engineering.

j / k navigate · click thread line to collapse