Yeah, but at least with PATs (not sure about other token types), you can't scope them to a particular repo, so whenever you need to allow something to even see a private repo or write to a public repo, the token you supply to allow that can do that for all repos and that alone is potentially really destructive. I am not sure if there is a good reason for why PATs can't be scoped to a repository, because if they were allowed to be, it would do a lot for security I think.

Most integrations just ask for blanket all permissions. They do this because it means they can give you a list of repos and let you choose which ones to integrate their service into with no work on your part except "click yes to give us permission to do everything for you and ... we'll do everything for you"