Let your Puppet/Ansible clear out all non-managed keys. If it's not in version control, you don't know who did what when. That's a nightmare as soon as you are more than two admins.
Also, the CA mode of OpenSSH is great. More people should use it. It's like PKI but sane.