Yep, and if you want to keep folders you didn't branch up to date with master you have to continuously rebase.
Not having it this way would be equivalent to having subrepo that refers to HEAD instead of a specific commit which is normally considered an big anti-pattern.
The best remedy is to not do branching like this in the first place, just try to stay on trunk all the time.