You are correct about the "one unscrupulous person" but VCs are hardly significant when you consider other options.
Every single aspect of business and government is susceptible to NSA snooping. Had I access and the lack of scruples, I'd much rather read selections from Goldman Sachs' gmail than any VC's; similarly I'd much rather listen to Obama's telephone calls than Paul Graham's. I'd much rather make my investments in the futures market based on e-mails from high-ranking officials in the DOD than on those from any social networking VC. I could make more money (or have more influence, etc.) that way.
As you say, it takes only "one unscrupulous person". So how common is unscrupulous behavior among highly-trained and highly-screened sysadmins? My bet, no less than 5 in every 100 would be unscrupulous, at least 1 in 100 would be absolute psychopaths, and possibly 1 in 200 would be unscrupulous enough and have sufficient social/business skills to take it to the bank in a big way (I'm talking serious crime/spying/nation-selling).
There may be "Snowdens" out there that we don't know about - persons who are now selling us (corporations, person, and nations) down the river for a few million dollars.
The NSA has put the nation in a very precarious situation.