The fact that it's okay to libel a public figure (a fact I didn't know until now!) further underscores how arbitrary it is. At what level of renown are you a "public figure"? How is it measured?
We should be as stringent as possible in legal exceptions to freedom of speech. But there's no moral obligation to protect any given speech from social consequences (such as getting kicked from your web host). No one is actually a free speech absolutist.