A "majority" of any sizeable group is not going to be rich. What is important for fiscal impact is that the rich are leaving, not that other people are also leaving.
If you want to see data, we can look at IRS data:
"New York’s tax base shrank by $19.5 billion while California lost $17.8 billion as a result of workers fleeing those states during a time when lockdown measures allowed employees to work remotely, according to the Wall Street Journal.
Other high-tax jurisdictions such as Illinois ($8.5 billion); Massachusetts ($2.6 billion); New Jersey ($2.3 billion); and Maryland ($1.9 billion) also saw an exodus of workers during 2020.
The states that reaped the benefits of the “wealth migration” include Florida, which gained an additional $23.7 billion in gross income; Texas, which gained $6.3 billion; Arizona, which took in $4.8 billion more; North Carolina ($3.8 billion); South Carolina ($3.6 billion); and Tennessee ($2.6 billion)."[1]
[1] https://nypost.com/2022/06/06/ny-cali-biggest-losers-as-weal...