That is entirely specious, pedantic and irrelevant. In the context relevant to hate speech, there is no doubt that, say, LGBTQ individuals or Jews are minority groups. I don't think you can really be serious about denying this (given that the numbers are what they are). I am not sure what point you are trying to make here.
>There is no specific connection to "minorities", whatever that means to you.
There is no essential connection between hate speech and minorities, but there's a very obvious connection. Historically, many victims of hate speech and other hate crimes have been members of minority groups. And it's not difficult to see why. It's a lot easier for a majority group to persecute a minority group than vice versa.