Well, but tomorrow there will be CTOs asking for a contingency plan if AWS goes down, even if planning, preparing, executing and keeping it up to date as the infra evolves will cost more than the X hours of AWS outage.

There are certainly organizations for which that cost is lower than the overall damage of services being down due to AWS fault, but tomorrow we will hear CTOs from smaller orgs as well.

No really true for large systems. We are doing things like deploying mitigations to avoid scale-in (eg services not receiving traffic incorrectly autoscaling down), preparing services for the inevitable storm, managing various circuit breakers, changing service configurations to ease the flow of traffic through the system, etc. We currently have 64 engineers in our on-call room managing this. There's plenty of work to do.

Well, some engineer somewhere made the recommendation to go with AWS, even tho it is more expensive than alternatives. That should raise some questions.

> Easiest day for engineers on-call everywhere

I have three words for you: cascading systems failure

Can confirm, pretty chill we can blame our current issues on AWS.

and by one I trust you mean a bottle.

/ai why is AWS down? can you bring it back up

After some thankless years preventing outages for a big tech company, I will never take an oncall position again in my life.

Most miserable working years I have had. It's wild how normalized working on weekends and evenings becomes in teams with oncall.

But it's not normal. Our users not being able to shitpost is simply not worth my weekend or evening.

And outside of Google you don't even get paid for oncall at most big tech companies! Company losing millions of dollars an hour, but somehow not willing to pay me a dime to jump in at 3AM? Looks like it's not my problem!

Qwen3 on lm-studio running fine on my work Mac M3, what's wrong with yours?