For example, I am more of the "All you need is Postgres" kind of software engineer. But reading all those fancy blog posts on how some team at Discord works with 1 trillion messages with Cassandra and ScyllaDB makes me envious.
Also, it seems that to be hired by such employers you need to prove that you already have such experience, which is a bit of a catch-22 situation.
In other words, the developers you're envious of didn't start with Cassandra and ScyllaDB, they started with the problem of too many messages. That's not an architectural choice, that's product success.
Overengineering is more prevalent the more money a company makes and companies who overengineers will pay good money to keep the overengineering working.
Unfortunately the rest of the executive has leaned on them so hard about AI boosting productivity they aren’t able to avoid thst becoming a mess