undefined | Better HN

0 pointsmarcinzm7y ago0 comments

Fair point but still seems odd that the people providing updates took the weekend off during a large scale customer impacting issue. I'm sure all the people spending the weekend trying to mitigate the impact of this on their infrastructure would love to have timely updates.

0 comments

11 comments · 2 top-level

izacus7y ago· 7 in thread

It's weekend, why wouldn't you take it off? It's just silly software.

DoreenMichele7y ago

A quick search suggests that silly software is used by dozens of companies classified as hospitals or healthcare.

Let's hope you don't have a life threatening medical emergency that can't wait near an affected healthcare facility while that silly software is down.

https://idatalabs.com/tech/products/kubernetes

steelframe7y ago

> Let's hope you don't have a life threatening medical emergency that can't wait near an affected healthcare facility while that silly software is down.

If your ability to operate an ER is dependent on a remote data center, you have no business being a public health provider.

ramphastidae7y ago

A lot of people's businesses, reputations and livelihoods depend on "silly software," not to mention that they are paying customers themselves.

Twirrim7y ago

I know sarcasm doesn't translate well on the internet, but how the heck did you manage to miss it in OPs post?!

1 more reply

jschwartzi7y ago

More to the point, why would you depend on Google for any critical infrastructure after this?

user59944617y ago

Because you tried to run your own infrastructure and it was so much worse.

Draiken7y ago

Then in a few months from now, people will be saying the same things after an AWS or Azure outage.

Am I the only one that finds this slightly humorous?

tweenagedream7y ago· 2 in thread

This is a problem in the web ui creating additional node pools, there is a very simple workaround to use gcloud. What major impact are you referring to?

It's likely the fix is checked in and will start roll out on Monday.

Disclaimer: I work on Google Cloud and while I believe we could use more words here, this doesn't seem like a huge problem. It's embarrassing the the issue with the ui was shipped, and I'm sure this will be addressed in the post mortem as well as whether it could have been mitigated quicker than a roll forward.

marcinzmOP7y ago

>This is a problem in the web ui creating additional node pools, there is a very simple workaround to use gcloud. What major impact are you referring to?

Based on comments in this thread even gcloud is failing and so are other non-kubernetes services. Which may be inaccurate but there's a lot of people saying the same thing so maybe it is.

You're right however that the linked issue is only about the UI. So Google isn't even tracking the service distribution issue in it's issue tracker much less updating people on. I personally think that's even worse...

kenan_warren7y ago

It's not just the UI though. Resources have been exhausted, or that's the error I'm getting, in a lot of regions for both my work and personal accounts (GKE and GCE). Someone on the GCP slack also said they were getting similar issues from Dataflow so it seems to be widespread across products.

j / k navigate · click thread line to collapse