It’s complicated. Ultimately it’s rude to refer to anyone by their skin colour, ethnicity or racial “group”. There are almost no situations where it’s appropriate.