> It would have to be one of the four people with root.
Or anyone who'd ever gotten access to the computer, or installed a camera near it, etc.
The critical part of that answer though, is "one of the". The system fails if any of the individuals is be malicious. A more-robust system would require multiple malicious agents in various organizational silos (security, compliance, management) to fail.
> every time the private key rotates
Well, if I got in once it probably phones that home for me.
> you really can't scale up this particular mechanism and keep it secret
Well, it isn't secret. We know the NSA intercepts hardware to muck with it, when needed. Much easier even than planting something in your server room explicitly. Also, they wield NSLs compelling silence and cooperation. It's not like being discovered here or there would stop scare or stop them.
It would probably scale pretty well given that this is the extreme; most people just generate keys on the old debian box in the corner.