but does everyone agree on what hate speech is? That's the danger. You can just claim any opinion you don't like is hate speech. You can say endorsing a particular candidate is hate speech and those people can justifiably be censored; their views invalid (and in some places; justifiably killed).
It was once considered offensive, in many places a crime, to say homosexuality is morally okay or that the Bible should be translated into German and English or to say God doesn't exist.
There is no distinction between "Free speech" and "hate speech," because it requires you to qualify the former. There are exceptions in many countries, but they are for very specific things: child abuse and advocating specific violence against individuals.