Agree. That's also the dilemma with asking the user for his permission, it is very difficult to frame a concise question and get an educated decision there. So, better to only ask if the App
explicitly requests that permission sounds reasonable.
The prior threat-model was, that e.g. a camera/gallery app which may/may not have a permission to a users current location, also has access to the history of a users' locations just by scanning the images when showing the camera roll.
It frankly makes sense to create a separate permission just for this location metadata AND strip this data when no permission was granted, I believe everything else would be MUCH harder to explain the user...