I deliberately don't accept promotions (to an extent) or respond to recruitment requests for any high level or management positions for that exact reason. I'm just another cog and I like it that way.
Logistics is tough. I have cycled through a few positions to find the right mix of meeting schedules. I try to work for low touch/low meeting culture companies that let me focus on my work. If there is an occasional meeting overlap I ask to reschedule due to personal conflict and there hasn't been a problem so far. Luckily remote friendly companies are low touch and tend not to micromanage my time.