This is actually the main thing I like about Knuth. Things are either proven, in very strict terms, or they are empirically studied. Also typically in rigorous means.

People usually present his work in terms of proofs only. Typically with big O considerations. Reading him, he very quickly warns of the dangers in big O analysis. (He is still a fan of it. Encouraged it as a math aid for grade school work, at one point.)

I recommend anybody to look at Dijkstra's notes on structured programming¹. It's a very worth teaching on how to conduct impartial research and transform a field of knowledge.

But TL.DR. - He got digged plenty of repeating cases of bad code, and proceeded to fix every one of them with very few coherent and systematic changes.

1 - His notes are here: http://www.cs.utexas.edu/users/EWD/ unfortunately, I don't remember what numbers to look.