If you want to audit mailed out ballots, use two envelopes instead. The outer one will carry a unique id, the inner one will be completely anonymous and contain the actual ballot. All the auditing can be done on the outer, unopened envelopes. Then they are stripped off, all the inner envelopes are thrown on a heap and only then opened and counted.
Of course, even then, the paper ballots contain traces of the voter's DNA, so the truly paranoid will not be happy either.