Precisely. I can't speak for all pacifists, but many, like Tolstoy, outright reject personal self-defense, at least in theory. That is to say, while it is very difficult for man to give up defense of oneself and one's family (probably for good reason), a pacifist does not see the self-defender as morally pristine. The great danger is when self-defense is given priority over other means of resolving or fleeing violence. If you believe the state's case, then I think the Zimmerman murder trial is a perfect example of zealous self-defense.