The problem is that slippery slope is not a hypothetical argument in this case. You can be jailed in France, today, for wearing a pro-BDS t-shirt, because it's "hate speech". At the same time, for all the political censorship that Europe has compared to US, it doesn't seem to be doing anything to stem the rising popularity of their extremist parties.
(https://theintercept.com/2017/08/29/in-europe-hate-speech-la...)
The problem, as usual, is that there's no "we" in "must we allow". You're not actually the person who's going to be doing the allowing - best case, if you live in a non-corrupt democracy, you'll be voluntarily giving that authority to somebody else; and not even to a specific person, but a role. If the bounds of that authority are too specific, it becomes useless in practice because it's easy to avoid its scope. If they are too vague, and the precise bounds are left to the discretion of the enforcers, it just gives them another channel to manifest their biases.