try it: take gaussian white noise with zero mean and integrate it twice. You'll see the signal does not stay close to zero, in fact it will drift arbitrarily far away from it over time (it's only necessary to integrate once for this to be true, but doing it twice as an IMU needs to will make it more obvious).
You are correct, I initially did not explicitly separate the noise according to its frequency (my mistake).
Integration only helps with high frequency error and can actually worsen low frequency error, more details in my second reply to kens.
