You can’t reach a program that solves the general case by iteratively writing a failing test for a specific case, making the smallest change required to make that test pass, refactoring, and then moving on to the next test. The important step — what you called “fully implementing the add function” — is taken by implementing some other insight that is not driven by the tests. That step isn’t a refactoring either, because it very much does change the behaviour of the program.

To design and implement a good program, sometimes you just have to know what you’re doing. There is no substitute for understanding the problem you are trying to solve and how you intend to solve it. And if you have that understanding and you design and implement your program accordingly, what is the value of following the red-green-refactor process compared to simply writing any unit tests you find helpful for verifying your implementation?