As opposed to something which can be smuggled out and reused offsite.
I'm also thinking that by centralizing (still locally) the configuration, we can get better key rotation hygiene habits without needing to compromise on credential granularity .
Just like there are security benefits in using a secured HSM instead of a world-readable private-key file stored in your unencrypted home directory, even if, yes, the HSM can be abused by a locally privileged attacker.
(I'm definitely not saying I have a silver bullet though, and I don't think one exists. Like any realistic solution, it should be part of a defense-in-depth strategy. Things like hardware keys make for incremental gains, etc)