Well if your primary concern is a local threat - which it absolutely is not for the vast majority of people - then you just have to be more careful with your keys. If you suspect someone might be actively trying to break in to your home, you wouldn’t leave your keys on your desk while you went to lunch.
Yep. Use FIDO2 keys to require a PIN or fingerprint to activate the key. This is why android/ios as a FIDO key is great - easy to lock, so built in two factors.