Sure, and it will be caught by CI (if you set it up (properly)) and code review (if it happens (and nobody misses it (and no commits are pushed as a cleanup after (...)))).
So basically if everything works perfectly, you won't have that problem. We know processes don't work perfectly, so it's better to avoid that issue in the first place by never leaving the literal tokens in the config.
Also, even if your CI catches the problem, it's too late - your credentials are now in the repo and in the CI system and likely in your log collector attached to the CI. You'll need to roll them. Same with reviews.
