I don't know about you, but my social graph is pretty diverse. I wouldn't trust all those very different people to make an important decision like "what sites should be visible to me in search engines."
You're right. Others should not be able to hide sites from my search results, but their decision on blacklisting sites can affect the ranking of sites in my search results.
Nobody said you had to trust everyone in your social graph. I definitely wouldn't, but that doesn't mean there aren't a few select people that I would trust.