It's not fair to raise taxes on other people who are honest and pay because it's a pain to collect from some people.
I realize that's simplifying things some, and not all people who don't pay do so because they're sneaky or dishonest or whatever, but I think the broader point stands.