Would it make you happier if I said 1/3? Is that a reasonable fraction of your potential userbase to leave with a bad impression of your business?
You make it sound like supporting Internet Explorer is difficult. If it's 2003, then yes, I'll agree with you. But today it's not.
Supporting IE in 2011 is a matter of building your site against Chrome while not going out of your way to use non-standard things like Object.keys, localStorage or Canvas when you can avoid them. Then you test on IE7, notice your corners aren't rounded, and decide you can live with that.
If, on the other hand, you go out of your way to cram in as much HTML5 nonsense as you can fit, or refuse to spend a few minutes getting IE to render, you need to realize that you're not taking a stand. You're just being lazy.
But then at the end of the day it's your business, so you're well within your rights to be lazy. You may or may not make less money as a result, but then again that's nobody's business but your own.
All the best.