I think my problem comes down to my point about living in a broken home. If you have broken windows, fix them. If you just have windows you don't like, do a cost/benefit on replacing them.
And realize that, piecemeal replacing things may not be a good idea. Running with the window analogy, if all you did was replace one window. Sure, in some way you are better than if you had not replaced it. Odds are high it will actually hurt your valuation on the home if it doesn't match the look of the other windows, though.
Same for the car analogy. Driving a low MPG car is ultimately expensive. Unless you have done a cost benefit to getting a high MPG one, it is likely not a good idea to upgrade.
This is especially true for most nonsense articles on paying technical debt. They almost always involve becoming an early adopter of a technology. Something which is known to be expensive elsewhere. Consider, the number of folks that saved money by buying a first model or so prius is probably zero.
At work, this is especially poignant. The folks that were most into paying down technical debt have managed to land us with no fewer than 4 dependency frameworks. Sure, I hate struts as much as the next person, I would welcome that to the frankenstein we have wound up with.