You also need to know the exact implementation of your compiler's optimizer, otherwise you don't know whether it's safe to change your program. Any change might give the optimizer enough information to unleash the nasal demons in previously cromulent code.
And it's stuff like this which is making me increasingly uncomfortable with using C++ as my main language. Currently it looks like Rust is the most suitable replacement but I'm not sure if it's there yet.