What’s a ‘luxury neighborhood’? In any large city, most central areas are generally a bit of a mix; “this is only for rich/poor people” is more of a suburban phenomenon. If the shops are only catering to the high end, a problem can develop.
(I am less sure why this happens to such a great extent in large US cities vs elsewhere, vs more of a balance of shops, but it does seem to.)