The use case is ssh keys! If malware can run an ssh command on the remote host, it doesn't need to steal your key, it can just install itself there. Or add its own keys to the access, etc... At best, you'd have to detect and fix that sort of thing with auditing and control, something that's isomorphic to the "third party" requirements I was mentioning.
To repeat the third time: this is all terrible threat model analysis. TPMs do not have value for individuals managing access between trusted devices. TPMs are for third-party validation.