undefined | Better HN

0 pointsmaerF0x06y ago0 comments

Actually OP gave the exact reason ECS does not work for them:

> The rate at which we can start tasks restricts the parallelism of our deploy. Despite us setting the MaximumPercent parameter to 200%, the ECS start-task API call has a hard limit of 10 tasks per call, and it is rate-limited. We need to call it 400 times to place all our containers in production.

That call limit needs to scale with the cluster size

0 comments

3 comments · 1 top-level

otterley6y ago· 2 in thread

That sounds different from the usual way customers perform an ECS deployment, which is to replace the task definition of the ECS Service with a new one and let the control plane manage it. Only a single API call is needed, and the control plane can launch replacement tasks pretty quickly itself.

ahmcb6y ago

We don't actually call the startTask API ourselves, but when we tell an ECS service (using cloudformation) to use a new task definition (which is basically just a new ECR image tag), ECS calls startTask and various other APIs on our behalf.

otterley6y ago

That is true. FWIW, as an experiment I built a 50-node ECS cluster and launched 4000 tasks on it. I couldn't build a single service with 4000 tasks due to soft limits, so I built 4 services with 1000 tasks each (nginx, 256MB, 128 CPU units). No load balancer.

Overall task replacement time was about 15 minutes -- which is pretty respectable, in my view. Obviously not as fast as doing a hot code swap, though. Tradeoffs abound.

j / k navigate · click thread line to collapse