Because the keys never exist "on disk"? Why isn't every password manager pwned on every persons machine is what you're asking it seems.

A process can not dump the memory of another process if those processes are executing under different users, or the process performing the dump is root.

On many OS's there are even more strict restrictions, where within a user a process can only dump the memory of processes that are its direct descendants.

they would have access to the socket not the key, sure a very elaborated attack can probably figure out how to exfiltrate a lot of things (since they have already compromised the host) but for most, if they don't see things in ~/.ssh they would just go away and figure out another host to exfiltrate keys