No, they pay more because they can afford it. When a bidding war starts between a Californian and a Texan for the same house, the Texan loses every time because they can't bid as high as the Californian.
It isn't just housing, either. When a new grocery store opens up in one of these neighborhoods, do you think they just sell their goods as cheap as they always have? Of course not. They want to make a profit, so they up their prices because they know the Californians are more than willing to pay for it. The Texan who now struggles to make ends meet because food is more expensive? They get pushed out.
> You can say "Americans are exporting higher cost of living" by purchasing avocados from Mexico, or factory capacity in China, etc. and depriving the locals who can't afford to pay the same.
Yes, that is exactly what is happening. Have you ever been to any of these countries where Americans-with-money swoop in and starting buying up goods or property? They hate it, just as much as Texans hate the Californians who are moving in.