The problem is purely with how some companies are applying SMS as an auth factor. In cases where SMS us being used as a recovery factor, it should not be allow for immediate recovery. Instead the user should be notified via other channels (email, phone notifications) about the recovery attempt, be given the opportunity to reject it, and for the recovery to only succeed if it is not denied after e.g. 3 days.