In my experience, that's really the only way to do software. It's completely unrealistic to think you'll get it right the first time, and yet completely destructive to go and start from scratch. So you pick your worst problem, rewrite it, then hook it back into the working system without touching the rest. Then pick your next-worst problem, rewrite it, etc.