Private and charter aviation exists and is free from those constraints.

Certainly not unreasonable. But it does require you to commission your own transport subject to the rules that that private entity seeks to impose. Public entities which indiscriminately service residents and visitors of a given territory would obviate this requirement. But if you're in the US, good luck convincing taxpayers to agree to pay for that.