That's the question. I think it depends on why you didn't want to install the Google Services in the first place. I mostly didn't want the remote wipe and remote code execution privileges the Google Services come with. It wasn't primarily a question of privacy as such.

I doubt the security problems are huge. Most userland apps run as a dedicated user. Security problems with the Google Services are potentially much more serious.