Hmm, I don't think it's that purist. All my HTTP clients have the logic that 4xx is a non-retryable error, i.e. if I repeat the request I will always get the same response unless something is changed, whereas 5xx means that something is wrong on the backend and I should retry it later.
It is true I have often encountered inconsistencies in implementations but I do think that when that happens it really is a backend bug that should be fixed.