The 'Fire' example is overused. The speech alone can directly cause physical harm. Racist, xenophobic and other distasteful speech does not. Free speech, in the USA means you have a right to distasteful speech, actively harming someone with speech is just as bad as harming them with a stick, therefore it can be prohibited.
----------- End Nitpick --------------