Usually businesses catering to the upper classes don't usually do so at the detriment of "the poor". For example Ferrari selling 100k+ cars doesn't prevent me from buying a cheap car. Homeowners renting to "rich people" prevents a poor person from renting the same apartment.
This could of course happen with regular rentals, but the difference is that nobody would pay above market rates for a long-term rental, which is what basically happens with Airbnb.