That is very much the point, I think. You can't judge the speech negatively on its merits, since emphatic negation is a regular feature of many languages. You can judge the speech negatively on the basis of
who you think the person is, in this case "black".
There are some obvious issues with confusing these things.