The truly paranoid will implement an SSH CA that will sign the key's cert for a limited length of time, using u2f or similar to authenticate the user at the time of signing.