We have laws, but that mere fact is not an excuse to make any and every law so it is not a very strong point. Besides, free speech is law, and it is constantly enforced. It is law. And it was a law very intentionally created so that we can have a functioning society, explicitly made from the perspective of someone who was subjected to restrected speech from another government. Free speech is a rare liberty that only the most prosperous nations enjoy, and had to be payed for with millions of bodies and blood. I highly recommend talking to an older person who fled the soviet union.
Side point, I do not believe in hate crime as distinguished from crime. Crime is crime. If I murder someone is it twice as immoral if they are black? There should be no distinction. The mere existence of a legal distinction violates my good sense.