Although I may have been more frustrated than most by paramiko's performance and stability.
I've started thinking of server configurations in Ansible terms (i.e. "I want X module installed, Y directory present, etc"), but sometimes nothing beats a simple bash command to git pull && sudo service restart.
Have you tried it since they switched to using ssh by default (instead of paramiko)? Because in my experience Ansible can ship multiple modules faster than Paramiko can connect to a host. And it is vastly easier to build reliable tasks out of modules than bash scripts.
http://www.ansible.com/blog/ansible-performance-tuning
If you want to use paramiko (like Fabric does), you can -c paramiko too. This mode came first, but -c ssh offers more features and can be just as quick.
EDIT: nevermind, found it:
The reason you want a fully created ami is to greatly cut down on the time it takes to accept new traffic. Lets say you get a spike in traffic, if you have a fully baked ami your autoscaling group can get an alert of the increase in traffic, launch a new instance and that instance will start handling traffic around 60 or so seconds. If you booted a generic ami and then provisioned the box you could easily be looking at 10's of minutes before it could handle traffic depending on the complexity of your app. This could easily lead towards long periods of down time.
What you don't want to do is attempt to scale, and you can't reach Github, or you overwhelm your configuration management tool, etc. Your AMIs should be built so they need almost nothing from external resources before being ready to serve requests.
- Gzipping the subset of code that needs to be deployed
- Uploading that tar to S3 and creating some infrastructure so that new instances pull the current from S3 on boot
- Creating a fab script that ssh's into our instances and re-runs the on-boot script to point them to a new deploy versionInstead of Ansible, I used a plain old shell script, but still idempotent. I did all the testing locally using vagrant/virtualbox. Application deployment was separated in our setup as well, also retrieving the latest application codebase during boot, but using a tgz stored on s3 with the application all the application dependencies (ruby gems and precompiled assets) already in there.
All in all I am very happy with this setup being able to autoscale in a few minutes.
http://zwischenzugs.wordpress.com/2014/08/09/using-shutit-an...
This deploys in a reliable way using a docker container and ShutIt, a tool we use in my corp for fast development of reliable and composable iterative steps:
http://ianmiell.github.io/shutit/
Here's more on the subject:
https://github.com/ianmiell/shutit/blob/master/README.md https://www.youtube.com/user/ianmiell http://zwischenzugs.wordpress.com/