undefined | Better HN

0 pointsverdverm6y ago0 comments

Make golden images with packer, or something similar, and then roll your fleet over.

You should not be running package managers on production servers. Or any of the other things salt, ansible, chef, puppet can do.

0 comments

yjftsjthsd-h6y ago

As in, no human should run a package manager in prod? (But salt/ansible/etc. running it is fine) Same idea as "if you're SSHing to prod, something is wrong" (where provisioning tools make all changes, logs are all aggregated and delivered in their own tool, and even debugging is built into the app or logging system).

verdvermOP6y ago

More as, you should not modify images running in production. By human or machine.

Rather build new images and roll over the fleet. If you need to debug, remove from production (quarantine) and work on it there.

Don't run master / agent setups for ansible / salt anymore. You can still use them for creating images, which are later turned into running VMs. Think about it like containers. Do you update the contents of your running containers, or log into your containers to make changes?

Better yet, use OSes that cannot be modified.

Spivak6y ago

But golden images on Linux are, well, messy. It's very annoying to make a clean VM template without some post-provisioning like cloud-init. And for most shops if you're running cloud-init you could do that post-provisioning with Ansible or Salt. And since your images are built with Ansible/Salt in the first place you might as well just build each VM fresh and use the vendor's ISO. One less thing to maintain and update.

Plus when you're in a pinch, which never happens of course, you can make changes without having to roll your VMs.

I feel like Atomic distributions are basically the happy medium between the two worlds.

1 more reply

bifrost6y ago

I can tell you, this is not something that is rare (humans in prod). Ansible often enables this behavior rather than removes it.

j / k navigate · click thread line to collapse