A lot of voting, Brexit and other stuff, comes mostly to whether people are pissed off with the current situation or not. If pissed off they vote for a change as in Brexit, otherwise not. I'm not sure a lot of the reasoning gets much deeper than that.
I remember asking Brexit voters who said they didn't want EU regulations to name one actual regulation that inconvenienced them and I don't think I got one answer.