The second layer is the downside, it makes it annoying to log into stuff. Even using SSH keys is awful. The UX of being able to log into things from wherever on whatever device because you remember your password is unparalleled. Yeah you have to worry about being keylogged which like, sure, it's a worry, but I've spent a lot more of my life being annoyed that I couldn't log into something because I didn't have the SSH key on that machine or trying to find my stupid yubikey etc than I have dealing with the afermath of having something hacked.

(which to my knowledge only happened once when I set the root password on my VPS to 'toor' before i knew about internet background hacking radiation, and sure maybe i'm compromised right now, but it's absolutely not affecting my day to day life so i'm not gonna worry about it)