Exactly. There is a rule but I would venture that very few native English speakers know it or were taught it; it's learned implicitly instead. I only found out there was a rule because I stumbled across the phenomenon doing a computational linguistics project.
My broader point (which I made poorly) is that native levels of language proficiency require a mix of explicit and implicit learning. There are certainly explicit rules for verb conjugation and agreement. Some things can be learned either way, like adjective order. Others seem to be mostly implicit: I can't give you a general rule for why "I asked her to marry me and she agreed" sounds fine but *"I asked her to marry me and she concurred" sounds weird.