I think there are a ton of very well-spoken arguments for why everyone, yes even those who "do nothing wrong," should care about privacy, and I would encourage you to look at those. Mostly because the people who publicly speak about this are much more eloquent than I, and have thought much more on the topic.

That said, I think it boils down to this: we all want privacy, but digital privacy is a hard problem because we are not "wired" to understand it. Humans are not good at interacting with systems with perfect memory, huge computational power, and extremely insightful statistical modeling capabilities.

This affects us differently than interacting with a person. Now, a machine can categorize you automatically based on political beliefs, religious beliefs, friend networks, conversational style, etc. This can be used to target you for (arguably unethical) influence via surgically targeted propaganda, or to retroactively mark you as a dissident in a tyrannical regime based on a comment you made off-hand years before the regime took power.

Left unchecked, these invasive tracking systems could be used for a myriad of unethical purposes. And even if you "do nothing wrong," it's important to remember that in most legal systems, it's very difficult to lead a normal life and never break any laws. Add to that the very, very long records these systems are capable of keeping on you, and I think it's clear why many people, myself included, wish to minimize our presence on platforms such as Google.