I am often quite baffled by people using only the device.
The whole point of all this is "something you have, something you know".
Yet lots just have passwordless keys for ssh with their yubikey. Completely unsecure, unsafe in examples you cite, and more.
When using ssh keys for login, you should enforce remote/server password requirements and an ssh key. This is trivial to do in sshd_config, and important.
Never trust end users to have passwords on their ssh keys. Always enforce it server side.
