Warm pools are a supported feature in AWS on auto scaling groups. Works as you're describing (have a pool of instances in stopped state ready to use, only pay for EBS volume if relevant)
https://aws.amazon.com/blogs/compute/scaling-your-applicatio...