How do you know when is the optimal time for those revisits? Is it a week later, then a month later, then 6 months later? How do you make that decision? What method will you use to revisit? Will you locate the original book, find the right page, re-read the passage? Or perhaps you'll go to the right section in your notebook and re-read that?

Any time you do all of that you're essentially doing what Anki does but in a less efficient and less automated way. The point of Anki is (a) to make a reasonable decision on your behalf about when to revisit, and (b) to automate the process of retrieving and displaying the information which is being revisited.

I've considered and accounted for it, but if I remove the spaced repetition part, it all falls apart, whatever the variation happens to be. At best it works in the short term, but after that, much of it becomes vague background noise.