It is a failing of the ego to think that one has foreseen all possible abuses of a policy going forward.
I think the answer is more speech, not less. Any exception you carve out will be abused in the future, based on the history of humankind and the behavior of governing entities throughout.
Take these exceptions to freedom of speech, add to them a codified framework for equity (which some are pushing for), and you're laying the groundwork for a society like that seen in Harrison Bergeron.