Already occurs - The word Asian is used for people from East Asia, as well as people from anyhere in Asia regardless of which specific part with no regards to ethnicity.
The word 'Aboriginal' also means Indigenous Australians, with an alternate meaning the indigenous people of a non-specific land.
I'm Asian myself. Might I get annoyed with the word? It's a possibility in another universe. Would I be annoyed by people who use it because they use it differently to what I expect? No.
English is just another language. In javascript you might be annoyed scope for variables is different depending on whether var is present. Are you going to be annoyed at people who use the presence or absence of var to designate whether the variable is local or global, instead of using window.myvar exclusively for global variables?
Well that is your choice, and personally I'd rather be fine with it than to get angry at anyone who use the word Asian or Aboriginal or write javascript differently to what I expect, otherwise I'd be angry a lot, making myself and others more upset than is necessary.
Anyways it was my attempt to empathise with both of them.