Well, my point was based on the assumption that there is a flaw/backdoor in RSA, but nobody has discovered it yet. So you'd just need a model that can take a textual description of the algorithm as input, and spit out a list of logical errors/flaws.
I'm not saying there is a flaw or backdoor, but if there were, then a LLM would potentially be able to find it, while a team of human experts could miss it.
A model that receives an RSA encrypted payload and outputs the decrypted version would of course be impossible unless the above assumption is true... or you give it access to some compute power so it can either try to brute-force it, or try to track down and hack the servers with the keys :P