This is the exact use case I have, and the frustration of not being able to select what I want made me create the app. I'm really surprised this functionality doesn't come with the Mac, but I couldn't find a workaround that didn't involve a manual switch somewhere.