This is not really that different than the spam "debate" - I've heard people argue that spam is no big deal because the bulk of it is caught. Tell that to people who run mail servers (but only if you brought your earplugs).
I suppose that, because many people put up with so much surveillance, they have difficulty drawing a line. I find this one a simple line to draw, but if you feel the need to place it elsewhere, the best predicate tipping point is based on intent.