The guys who wrote the universal declaration of human rights seem to disagree with you.
"Everyone is entitled to all the rights and freedoms set forth in this Declaration, without distinction of any kind, such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status."
"All are entitled to equal protection against any discrimination in violation of this Declaration and against any incitement to such discrimination."