That'd at least mean an attacker would need to be MITMing the connection close to your server/loadbalancer instead of near to the CA?
Wouldn't help if someone was doing a Quantum Insert style attack from within your hosting infrastructure. (I wonder if there's pre-build AMIs to let you quickly spin up open source implementations of Qantum Insert on AWS already?)