undefined | Better HN

0 pointsprmoustache1y ago0 comments

That is not answering the question.

How do you do to get alerted of your service going down when:

- you are in the wilderness, ouside or with little cell coverage

- drunk and dancing in a wedding with music at full blast

These are just 2 small examples out of many others. Also it means you need to stay connected when taking a plane, be able to stop and/or swap drivers if your are in a long driving trip so that you can fix your service in a rest area or while your partner is driving, etc.

I am pretty sure outages are very rare but if that happens the day your are out of cell coverage and unable to react, you might lose a lot of trust from your customers.

I am also suprised he relies on only one hosting service. I would have thought you might not want to have everything in the same basket.

0 comments

ozim1y ago

Keep in mind that 95% downtime happens when you deploy things or change configuration settings.

If I am drunk dancing in the woods as a solo operator no one is doing config changes on my servers.

The remaining 5% downtime like internet connection to the server facility, solar flares, UFO taking over the world - I would not able to do anything about anyway, would have to wait until it goes away.

martinald1y ago

Yes, I agree entirely. The amount of times that I've thought a particular service I've made is "reliable" because of great uptime for 2-3 years (with no changes), that I do a small innocuous change and cause some issues really makes me remember this.

The most common issues I have with long term projects are small code/config changes (#1 by a ridiculous degree), then way below is drives filling up and then way below that is actual hardware failure.

Drives filling up are often the most annoying to fix as they cause very odd user facing problems. Just fixed this week an old wordpress site that I spent over 2 hours trying to fix, trying everything, before I realised the mySQL database was full. It turns out if a mysql db is full the provider revokes insert privs on the user and wp login will just go in a loop back to the login page if it can't write to the database. No errors on the UI :(.

_xnmw1y ago

Even Google has outages sometimes. The chances of an outage happening at that exact moment when you are "drunk and dancing at a wedding" are probably lower than "catastrophic deployment failure at FAANG, multi-hour outage", especially because you are certainly not deploying any changes at that time. And if the outage is due to a third-party dependency failing then you can blame it on them and there's not much you could do even if you were online anyway.

tpetry1y ago

The most important part is understanding that downtimes are not a problem!

Cloudflare is again down and your service is not available for an hour? You cant do anything. And nobody unsubscribes.

Most downtimes are because another service is down and you cant do anything.

Sometimes its really your issue. Your service is down for an entire day? Some customers may be pissed at the moment. But in the end they dont care if you are not a critical service - like this one.

Learning: Downtimes are not an issue. When such a message pops up just ignore it and look at it when you have time. This is strange first as we try to be perfect. But the world will continue to turn.

j / k navigate · click thread line to collapse