I feel this. Because of a Google Group I briefly followed one of my email addresses got incorrectly associated with my Kickstarter account on some marketing list somewhere and gets added to so much "legitimate" marketing lists for fly-by-night Kickstarters. It's really frustrating and the accident of it being a "wrong" email at least makes it somewhat easier to manage (though I worry if I ignore that mailbox too much I may miss the rare once in a few years important email to it).
For a while MailChimp was the only one of the major/reputable providers I trusted the Unsubscribe button on because they had a "I did not sign up for this button" that supposedly dinged the mailing list owner's reputation with them, but more than that would supposedly make it a bit tougher for the next mailing list to just dump that email in without a verification step or a cool off period.
That button disappeared recently and I guess MailChimp no longer cares either. Shame.