There's a hit to their public image too. When your home or car is burgled you don't want to see a tank.
And, as toomuchtodo hints at, heavily arming a police force leads to those guns being used. Ignoring "bad guys" for a moment, shooting innocent dogs of innocent people is probably a bad thing. Shooting innocent bystanders is probably a bad thing.
http://www.popehat.com/2013/12/05/nypd-baby-you-know-we-love...
Obvious Popehat bias, but still, it's pretty alarming.