> Political scientist Angelo Falcón argues that the use of broad terms like "person of color" is offensive because it aggregates diverse communities and projects "a false unity" that "obscure[s] the needs of Latinos and Asians".[39][40]
https://en.wikipedia.org/wiki/Person_of_color
Another criticism I've heard is that the terminology for POC originated among black activists during the civil rights movement in the US who were reclaiming the term "colored" used in the Jim Crow/segregation era, and so it's only really appropriate for black people to use.
The former term (nonwhite) is iffy too, but sometimes it's a better descriptor
To be clear, I haven't heard many people voicing the second criticism -- I think most people using "nonwhite" subscribe to the former point of view