The point of an election is to give a leader a mandate (https://en.wikipedia.org/wiki/Mandate_(politics)) to rule...
From a theoretical perspective, if a president wins by 1 vote, how much of a mandate to rule do they actually have?
Our governments founding document specifically mentions the consent of the governed. I consent to tax policies and housing policies I disagree with, but I don't consent to a government that doesn't seem to have any ruling philosophy other than might makes right. I definitely don't consent to a government that threatens allies in a way that irreparably damages the negotiating ability of future presidents, and I don't consent to rule by a person who wishes to make my future votes meaningless.
If you vote for someone that violates rule of law, it contradicts the system of voting itself.
I don't disagree we "elected" him, just that election is a very black and white view of politics. We elected him, but he does not have mandate to do what he is doing.