Yes. It's called "catastrophic forgetting". These models were trained on trillions of tokens and then underwent a significant RLHF process. Fine tuning them on your tiny data set (relative to the original training data) almost always results in the model performing worse at everything else. There's also the issue of updating changed information. This is easy with RAG - replace the document in the repository with a new version and it just works. Not so easy with fine tuning since you can't identify and update just the weights that were changed (there's research in this area but it's early days).

Again, that's why I said it is challenging.

I regularly do fine tuning on a model with fine results and little damage to the base functionality.

It is possible, but it's too complex for the majority of users. It requires a lot of work per dataset you want trained on.