Stripe was down (opens in new tab)

(status.stripe.com)

90 points1as10y ago20 comments

20 comments

I work at Stripe. We're back up now. Your customers can safely retry any failed payments. We'll post a public postmortem over the next few days with more information.

If you need any help, please drop us a line at support@stripe.com (and feel free to CC me; I'm michael@stripe.com).

kumarski10y ago

thanks for the responsiveness.

siliconc0w10y ago

"Elevated error rates" is now status page speak for "shit is totally down". Amazon does the same thing with a small note next to a green check when their api is also broken. I wonder what hellscape on earth would be necessary for amazon to drop to a red icon.

mentat10y ago

It happened during the DynamoDB outages but only for some subservices.

keehun10y ago

What I'm about to say is definitely NOT IN ANY SENSE Schadenfreude, but...

I get really excited when amazing services we've come to take for granted goes down. I'd put places like Amazon, Netflix, Gmail, etc in this category. The reason? Well, I really really look forward to reading the post-mortems and getting a sneak-peek at the inner workings of said services.

potatosareok210y ago

Don't be so excited! Maybe they just dropped another DB index and the guy who did it still has no visibility into the actual DB load.

https://support.stripe.com/questions/outage-postmortem-2015-...

breakingcups10y ago

Oh great, I thought I was the only one!

LeoPanthera10y ago

At the time of writing, this post is 11 minutes old, but the status page says they have been down for 5.

Hmm!

adrtessier10y ago

I think generally when every alarm bell in your monitoring system goes off the first thing you do is question whether monitoring is broken. When you confirm there is a problem this big, you panic and try to fix it really fast. Then you call your other on-call guys and tell them you actually have an "oh, shit" situation.

Once you recognize there's a serious problem, THEN you make the public announcement. Ah, the life of ops.

jfoster10y ago

They also don't seem to count the "partial degradation" periods as downtime; Stripe.js and Checkout.js both have a 9m red bar, but Checkout.js has had a lot of "partial degradation" periods. Both sit on 99.992% uptime, though. Maybe that's not so bad; it would depend on how severe a "partial degradation" can be and still be classified that way, I suppose.

ojbyrne10y ago

I tried ordering from doordash about 30 minutes ago (approximately 7:20pm PST) and my card got declined, which was presumably because of this outage.

nolim1t10y ago

For payment processing, I've built in a system which uses stripe (credit cards only) and bitcoin (using a different API. But that is only to detect whether payment has received. If the API is down, the detection of payments can still be done manually thanks to the fact that the bitcoin network is decentralized).

I never rely on a single point of failure. And of course, always good to have a "plan B" in place.

acconrad10y ago

all services are back online

gesman10y ago

Grinch's XMAS DDOS blackmail?

amateur_soclgst10y ago

No better time than Christmas to have your payment platform go down :(. I can't imagine the clusterfuck at the Stripe offices atm. Overnighters ahoy!

j / k navigate · click thread line to collapse