AWS: Network Connectivity issues affecting EC2 in US-EAST-1 (opens in new tab)

You should have duplicate infrastructure in another region which you fail over to automatically or manually.

I see the same thing. Pingdom reports higher response-time, but no downtime (meaning no alert). Also no alerts from AWS Cloudwatch. I first became aware of the issue when internal api-tests started failing at 9:56am CET. I see users accessing the site, but I don't know how many it's failing for.

Down vote? Really? I guess I should be thankful that I didn't have to explain this to a client: http://i.imgur.com/URf0H.png

Our app is also on Heroku and it did experience problems.

git push seems usable. :(

It does, but not for nefarious reasons -- it (until the more recent history) the cheapest region in the world for AWS -- it was only right before Oregon rolled out that Ireland, US-EAST and US-WEST-2 all became the same price point, but for the 4 years prior to that, it was always the cheapest so that is where most customers rolled out most of their infra.

Now that the prices has normalized I think the load is distributing more evenly, but for historical reasons I think that region sees a lot more churn (starting/stopping/deploying/etc.) -- just more grinding on the hardware at that region that others.

rkalla's answer is good, but there's also the issue of the US east coast being a good place to deploy applications that need decent performance to the majority of the english-speaking world. Dropping your servers there puts them within reach of Europe without as much RTT as the US west coast.

Obviously the right answer is to deploy applications into multiple AWS regions, but that's not appropriate for every service's architecture unless built that way from the start or modified specifically to do so.

In addition to the other two excellent answers, there's another reason for the apparently higher issue count:

US-EAST-1 is enormous. It's now made up of 10 data centers with tens of thousands of machines (and associated infrastructure). There are thus more moving parts, and more points of failure. While Amazon builds a bunch of redundancy into its systems, "smaller" issues will tend to impact a larger number of users in US-EAST-1.

Think so... it's the cheapest, and is the default choice. Sounds like it would have an order of magnitude more usage (and therefore problems) than the other regions.

Bullshit, I am still down.