Stripe's API was down again

132 pointsklinskyc6y ago56 comments

Hasn't hit their status page yet, but our error rates are spiked up again, and they just posted a tweet -> https://status.stripe.com/

5:35 Eastern -> Their status page has been updated

56 comments

43 comments · 13 top-level

dmlittle6y ago· 9 in thread

It would be great if Stripe would offer an RSS feed for status updates in addition to twitter posts. A lot of other services have one and it's very convenient to subscribe to in Slack for updates. At my current company, for example, we have a #third-party-outages channel which we have all available third party services (GitHub, NPM, CircleCI, Slack [yes, it's meta], HackerOne, etc.) status page RSS feed notify in case of an outage.

harrygallagher46y ago

It wouldn't be perfect, but there are tons of services that can generate an RSS feed for a specific twitter user. It looks like @stripestatus only tweets when there's actually an issue (aside from updates on current issues)

annon6y ago

I do the same, you can use the slack twitter integration to post the stripe alerts.

jammygit6y ago

I didn’t realize you could use slack as an rss client

ComputerGuru6y ago

Via a bot.

jakecopp6y ago

I thought RSS requires polling - to get timely outage notifications does Slack hit all the RSS feeds every minute?

leesalminen6y ago

I’ve used StatusGator for this in the past with success.

eropple6y ago

StatusGator isn't granular enough. Having a service go yellow just because a totally different geographic region got sad is annoying.

procinct6y ago

https://status.stripe.com/

Edit: Actually looks like that page doesn't have an RSS feed, never mind

leesalminen6y ago

I don’t see an RSS feed option there?

1 more reply

tibbon6y ago· 6 in thread

Stripe gets the front page because they are popular with developers, but I've dealt with so many other more-corporate payment processors that are absolutely awful for reliability. Rampant downtime, terrible support, terrible concurrency limits, failure to meet SLAs, terrible documentation (900 page specs that are impossible to read). They all think they are awesome, and yet can't go more than 48 hours without having some system degradation email sent out. I don't want to name them, but they are some of the worst businesses I've ever dealt with. They just want to sit there and collect huge amounts of money for helping you connect to Visa/Mastercard/etc, and do little else.

Rage. Stripe hitting a little downtime is nothing in comparison.

jchw6y ago

I have some experience with a Stripe competitor (may have predated Stripe by a bit) but their reliability was probably on par, more or less. Their API though... would’ve been worth a couple lost 9s for a better API. Polar opposite of Stripe on that front.

drspacemonkey6y ago

I dealt with one that seemed actively hostile. They would change their API without any advance warning at all. Data fields that we depended on would mysteriously vanish in the middle of the day, causing our entire system to grind to a halt until we put in a hotfix. Happened several times in the two years I worked on that project.

october_sky6y ago

Wow?! An API should be treated as a contract.. I hope you dropped them

1 more reply

personomas6y ago

So far I've had great experiences with Stripe's support.

joemag6y ago

Agree. I once had to explain the concept of idempotemcy to a payment processor, whose APIs lacked it.

robbyt6y ago

If this is true, why not name them to help others?

duxup6y ago· 4 in thread

Anyone have any good advice on storing a failed transaction and retrying later?

Is that even feasible with Stripe?

Obviously there would be security considerations.

ComputerGuru6y ago

The real solution is to never rely on one party. We fall back to another payment processor after using a heuristic to see if the error isn’t one that makes sense. If it’s a problem with the payment card, it’ll fail again, no harm done. If it’s a problem with the provider, it’ll go through.

(We wrote a payments abstraction library easy enough to (partially or fully) fill for any payment provider’s core functionality, so it’s literally just a couple more lines of code at the call site. It took several rewrites to get the abstraction to cover all the oddities each time we added a new provider implementation, though!)

maxgashkov6y ago

But for that you have to touch CC info directly with all the risks and compliance bullshit.

1 more reply

duxup6y ago

Thank you.

jijji6y ago

have your script test for your own network possibly being down, then if it gets a bad response, put the transaction in a queue to be retried later

jasonb056y ago· 3 in thread

Bad day to launch my new GAN book. damn it!

how_774623486y ago

I was able to still get through and get it.

jasonb056y ago

Thanks!

sauldcosta6y ago

We were literally just about to deploy an updated pricing model when our test suite started failing when using Stripe's API.

rglover6y ago· 2 in thread

Kind of incredible that this is the first time I remember Stripe being down in a long, long time.

Serious kudos to their team and hope they get this resolved soon :)

civicsquid6y ago

I've seen a lot of comments like this on HN since earlier today regarding the Stripe API downtime. It's very positive and I'm glad to see it in response to an event that usually garners frustration.

Never used Stripe's API myself, but it sounds like they are vastly reliable beyond these blips.

jammygit6y ago

Out of curiosity, why is this downvoted?

cristea6y ago· 2 in thread

The last error message on their credit cards endpoint reads «This API method has been temporarily disabled due to exceptionally high traffic». It might be a DDoS attack.

edwinwee6y ago

We're investigating what happened, but we know this wasn't a DDoS attack.

gr20206y ago

Ya - I'm seeing the same thing on the customer endpoint.

fbelzile6y ago· 2 in thread

It looks like it's up again for me.

pgm87056y ago

I've had transactions getting through every once in a while but most are not.

foobiekr6y ago

SAAS: if one transaction gets through, it’s not “down” or “unavailable”, it’s “elevated error rates.”

harrygallagher46y ago· 1 in thread

Ran into this one in the wild when it happened, before stripe status had even tweeted about it. I was trying to order doordash and got a "Could not deserialize JSON object" error when adding a credit card. Hope this is fixed soon, the girlfriend and I are very hungry :(

Edit: update if anyone cared, API seems back up and we have ordered food! :)

egty6y ago

I care! What did y'all get? Was it tasty?

slics6y ago· 1 in thread

Is it due to another Fiber Cut somewhere? Recently almost the entire east coast was down due to fiber damage. I guess the cloud services are as good as the wires that run in the ground.

jxramos6y ago

grounding the cloud!

edwinwee6y ago

We're seeing a spike in error rates and we're working on a fix now. (We've updated our status page: https://twitter.com/stripestatus/status/1149065544399609856)

dang6y ago

The thread from earlier today is https://news.ycombinator.com/item?id=20403774.

sauldcosta6y ago

Yep, definitely is. Have been getting 503s for the last 15 minutes.

paulmendoza6y ago

Second time today. This has been a painful day. That uptime percentage is really ticking down fast. The developer/test mode APIs also seem to be down now as well.

j / k navigate · click thread line to collapse

56 comments

43 comments · 13 top-level

dmlittle6y ago· 9 in thread

harrygallagher46y ago

annon6y ago

I do the same, you can use the slack twitter integration to post the stripe alerts.

jammygit6y ago

I didn’t realize you could use slack as an rss client

ComputerGuru6y ago

Via a bot.

jakecopp6y ago

I thought RSS requires polling - to get timely outage notifications does Slack hit all the RSS feeds every minute?

leesalminen6y ago

I’ve used StatusGator for this in the past with success.

eropple6y ago

StatusGator isn't granular enough. Having a service go yellow just because a totally different geographic region got sad is annoying.

procinct6y ago

https://status.stripe.com/

Edit: Actually looks like that page doesn't have an RSS feed, never mind

leesalminen6y ago

I don’t see an RSS feed option there?

1 more reply

tibbon6y ago· 6 in thread

Rage. Stripe hitting a little downtime is nothing in comparison.

jchw6y ago

drspacemonkey6y ago

october_sky6y ago

Wow?! An API should be treated as a contract.. I hope you dropped them

1 more reply

personomas6y ago

So far I've had great experiences with Stripe's support.

joemag6y ago

Agree. I once had to explain the concept of idempotemcy to a payment processor, whose APIs lacked it.

robbyt6y ago

If this is true, why not name them to help others?

duxup6y ago· 4 in thread

Anyone have any good advice on storing a failed transaction and retrying later?

Is that even feasible with Stripe?

Obviously there would be security considerations.

ComputerGuru6y ago

maxgashkov6y ago

But for that you have to touch CC info directly with all the risks and compliance bullshit.

1 more reply

duxup6y ago

Thank you.

jijji6y ago

have your script test for your own network possibly being down, then if it gets a bad response, put the transaction in a queue to be retried later

jasonb056y ago· 3 in thread

Bad day to launch my new GAN book. damn it!

how_774623486y ago

I was able to still get through and get it.

jasonb056y ago

Thanks!

sauldcosta6y ago

We were literally just about to deploy an updated pricing model when our test suite started failing when using Stripe's API.

rglover6y ago· 2 in thread

Kind of incredible that this is the first time I remember Stripe being down in a long, long time.

Serious kudos to their team and hope they get this resolved soon :)

civicsquid6y ago

I've seen a lot of comments like this on HN since earlier today regarding the Stripe API downtime. It's very positive and I'm glad to see it in response to an event that usually garners frustration.

Never used Stripe's API myself, but it sounds like they are vastly reliable beyond these blips.

jammygit6y ago

Out of curiosity, why is this downvoted?

cristea6y ago· 2 in thread

The last error message on their credit cards endpoint reads «This API method has been temporarily disabled due to exceptionally high traffic». It might be a DDoS attack.

edwinwee6y ago

We're investigating what happened, but we know this wasn't a DDoS attack.

gr20206y ago

Ya - I'm seeing the same thing on the customer endpoint.

fbelzile6y ago· 2 in thread

It looks like it's up again for me.

pgm87056y ago

I've had transactions getting through every once in a while but most are not.

foobiekr6y ago

SAAS: if one transaction gets through, it’s not “down” or “unavailable”, it’s “elevated error rates.”

harrygallagher46y ago· 1 in thread

Edit: update if anyone cared, API seems back up and we have ordered food! :)

egty6y ago

I care! What did y'all get? Was it tasty?

slics6y ago· 1 in thread

Is it due to another Fiber Cut somewhere? Recently almost the entire east coast was down due to fiber damage. I guess the cloud services are as good as the wires that run in the ground.

jxramos6y ago

grounding the cloud!

edwinwee6y ago

We're seeing a spike in error rates and we're working on a fix now. (We've updated our status page: https://twitter.com/stripestatus/status/1149065544399609856)

dang6y ago

The thread from earlier today is https://news.ycombinator.com/item?id=20403774.

sauldcosta6y ago

Yep, definitely is. Have been getting 503s for the last 15 minutes.

paulmendoza6y ago

Second time today. This has been a painful day. That uptime percentage is really ticking down fast. The developer/test mode APIs also seem to be down now as well.

j / k navigate · click thread line to collapse