We've written to SQS at rates several times this using only 3 EC2 nodes without issue.
The only issue we have seen is occasional and sporadic latency issues reading from SQS which causes the queues to backup a bit but it generally clears within 30 minutes or so.
Thanks for commenting!
What kind of application are you using with the SQS (language, purpose, usage)? It would be interesting to know for me :)
This test was only running for 50 minutes, maybe this could be the issue?
Best regards, -sorenso-
SQS itself may well be implemented on EC2 instances with auto-scaling (though I doubt it), but that's irrelevant to the test harness, which needs to scale out the number of nodes pounding on SQS.
The important bit is that SQS doesn't exhibit any of the scaling bumpiness of ASGs. It's a region-wide service, which you'd be very lucky to break on your own.
I'll be happy to answer any question about my implementation and testing.
In this test I'm using a standard SQS service i region EU-west. My integration with SQS is done from a simple application written in Node.js. My application is put inside an autoscaling group which scales up when the load on the servers is above my threshold. At the end of the last test I was pushing messages from 18 different servers at the same time. The request rate was above 6K/sec but the insert rate into the SQS queue only managed to peak aroud 2.3K message insert/sec.
I agree that this is strange. But I've also experienced strange stuff with Amazon ELB: http://litt.no/tech%20stuff/amazon-elb-latency-problems
Best regards, -sorenso-
- ELB takes about ~15 minutes to scale out completely when you hit traffic spikes.
- SQS is simple to use, rather cost-effective and quite reliable. It will add a bit of latency, but as the author notes, this can't replace Kinesis.
- The redrive policy on SQS is really nice to catch corner cases.
What is your traffic rate in request pr second? I would be interesting to know.
SQS is a simple and easy to use service. I really like it, but as you read in my post. I'm a bit concerned about the scaling. We are expecting to reach a peak traffic of 3K req/sec next year.
Best regards, -sorenso-
Please do a write up on your experience. I'll be happy to read about it.
Best regards, -sorenso-
