gcloud compute ssh <instance_name>
GCP takes care of generating a key pair. The GCP linux images configure OpenSSH and PAM to check you have relevant IAM permissions. IAP [2] is used in place of a bastion if the instance doesn't have an external IP address.[1]: https://cloud.google.com/compute/docs/instances/managing-ins...
https://cloud.google.com/compute/docs/oslogin/#how_os_login_...
It has more or less the same benefits that SSM has, and you can use the same method with ProxyCommand to establish authentication before connecting. You can also chain it using ProxyJump to use a bastion host. Wrapped in a tool like aws-vault you can quite easily work with 2FA and such.
[0] https://aws.amazon.com/about-aws/whats-new/2019/06/introduci...
I think the advantage of not having to have a public bastion is pretty nice though. I think i'd consider SSM over ec2-instance-connect
LDAP sends passwords in cleartext over an encrypted channel, I don't like the idea of someone else on the host I'm connecting to being able to read my password. Naturally you can use key based authentication with it.
I know that some people are required by law or regulation to log all these things, but personally I don't see much benefit of such audit logs. All bets are off when someone has local access to a machine. And there's this whole debate about using MitM-proxies for logging and security and the potential security implications they pose.
All that said, step-ca[0] and vault[1] looks really promising to provide some kind of provider independent way to authenticate in a secure manner. You also have teleport[2] but I haven't looked much into it.
[0] https://smallstep.com/docs/cli/ssh/ [1] https://www.vaultproject.io/docs/secrets/ssh/signed-ssh-cert... [2] https://gravitational.com/teleport/
1) We soon discovered that using bash (and not sh that SSM signs you into) doesn’t format the logs correctly and makes them unreadable 2) Using SSH over SSM doesn’t log at all
Both of these problems are on AWS github but as always, they don’t respond.
> The later would make any logging outside of SSM difficult There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of storing and auditing correctly.
Using the SSM tool as a ProxyCommand for OpenSSH is brilliant. I can SSH to an instance (turning on agent forwarding, etc.) or SCP files back and forth, all the same way that I'd do it with a regular host. But that host doesn't need to be exposed to inbound SSH traffic!
I was also able to get the ProxyCommand working in a Docker container, so I don't have to install the AWS CLI or the SSM plugin on my (Linux) host system, with an .ssh/config entry like this:
Host i-*
ProxyCommand docker run -i --rm -v $PWD:$PWD -w $PWD -u 1000:1000 -v ~/.aws:/tmp/.aws -e HOME=/tmp -e AWS_PROFILE <docker image ref> ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'
A few problems in practice:* We often use our bastions to do port forwarding, e.g. to connect to an RDS instance inside VPC. As far as I know, we can't accomplish this with just SSH over SSM, so we still keep our bastions.
* User management: SSH over SSM doesn't do any user or key management for you (which makes sense). It establishes the SSH connection and you're on your own from there. We use Keymaker [1] to dynamically create user accounts on the EC2 instances, and populate SSH keys according to the user's IAM profile. This works fine, but maybe there's room for simplification.
* Connecting based on instance ids (`ssh i-...`) can be awkward. It would be amazing if we could alias these somehow, like set up a CNAME that points to `i-...`. I wasn't able to get OpenSSH to follow a CNAME and then use the ProxyCommand. Maybe it's possible, though.
To make the ProxyCommand setup a little easier, perhaps you could use `aws ec2 describe-instances` with the appropriate filter? For example, the article I linked uses:
INSTANCE_ID=$(aws ec2 describe-instances \
--filter "Name=tag:Name,Values=$INSTANCE_NAME" \
--query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" \
--output text)
to grab the instance ID from the Name tag. You could set up a shell script as a ProxyCommand to do this automatically, although beware that ProxyCommand's %h will always lowercase the input...Note: the README says "I found myself in a situation where I couldn't easily copy a file from my machine to the server". Note that you can use S3 for that (`aws s3 cp ...`), though you should be careful with permissions on your S3 bucket
One thing that I found annoying through the web interface was the lousy support of special characters. Every now and then either some character combination or a big wall of text would make the browser-based terminal almost useless. I have yet to make time to configure the cli access which I don't think will have these issues.
Otherwise my developers can't get a Django shell in production for emergencies.