> I've just spent years wrestling with someone else's poorly written, ill-intentioned code, bringing it into line. I've taken the above approach of slowly reworking it. Sometimes I wonder if I just kept the tests and jettisoned large bits of it if I'd be better off?

That's the main reason we have this discussion in the first place IMO. There is no one right answer to the question.