When state is introduced, then CAP/PACELC distributed systems issues arise, and figuring out approaches to deal with them are a must. Read fly’s Postgres docs, and regions come right back in.
I point to Cloudflare (what I'm most familiar with). Across their entire stack of compute, storage, db, etc product lines you won't see a hint of region anywhere. It just works, everywhere. Same with Fastly/Oracle (although they like Fly are quite a ways behind Cloudflare).
I suspect (hope) that as Fly matures their apparent true goal of edge and region-less applications will mature more. Their compromises on region awareness for DB has always struck me as a more of a resource restraint compromise on their part than end goal. As you say these challenges are hard (expensive).
https://developers.cloudflare.com/d1/learning/data-location/
Hint Hint description
wnam Western North America
enam Eastern North America
weur Western Europe
eeur Eastern Europe
apac Asia-Pacific
The fact remains this is somewhat down in the docs because it is automatic and invisible to the point it works well enough where most developers don't know, don't care, and aren't exposed to it in any meaningful way. Contrast this with AWS where the first thing you are presented with is picking a region where everything lives, while not clearly and explicitly explaining that many things are ultimately dependent on us-east. Cue the shock and confusion when us-east has yet another issue and those in all other regions don't understand how and why they are impacted.
You are correct about the free lunch, information theory, and physics at the end. I don't think a reasonable person can think when you write data to a pop in Tokyo that it could possibly show up in Chicago instantly. That is fundamentally impossible and always will be. I think CF does a pretty good job making it "magic" for most devs and use cases while explaining these fundamental realities and limitations to those who need to know (potentially) for whatever their application is.
Here's the Durable Objects documentation [1]. Don't those "location hints" look like regions?
Also, here's the "Data location" page for Cloudflare D1 [2]. See "available hints" at the bottom.
I think it would be more accurate to say that Cloudflare's architecture is designed around having lots of regions, automatic migrations between them, and no guarantee about which one you'll get. But they're still there.
[1] https://developers.cloudflare.com/workers/runtime-apis/durab...
[2] https://developers.cloudflare.com/d1/learning/data-location/
I actually argued against adding these hints at all, since it confuses the messaging and most people don't need them... but they were easy to add and solved an immediate problem for a few customers, so we did.
In other cases we've held firm. For example, some people have wanted to restrict their Workers to run in specific regions because the Worker made lots of requests to a specific back-end in that region. Instead of actually offering explicit hints, we built a system to detect this automatically: https://blog.cloudflare.com/announcing-workers-smart-placeme...
In any case, most people don't have to think about regions on Workers, and for the few that do, our goal is to change that.
(I'm the tech lead for Cloudflare Workers.)
Only the most trivial business systems can operate without any state.
If your application has any sort of shared, mutable state, you absolutely need to pick a region, or go really deep into the EC/consensus/clustering rabbit hole. With a dedicated region with 1 big DB, you can achieve latency figures (over the shared state) that would be infeasible with other schemes. Serializing transactions in a distributed manner is a circus compared to what Postgres has to do to achieve the same.
For better or worse, sticking a big SQL database in exactly one region will almost always be the best path for most forms of business. If latency/edge are still a concern at this point, that's when we maybe start talking about more specific geographies and standing up read replicas in those regions. In my experience, if you are peddling B2B webapps, no one will ever complain about east vs west coast latency unless you actually screwed something up in code.