I posit that complex NN models can achieve the same level of interpretability as logistic regression. In part, because some interpretability methods use logistic regression as a white box model to explain black boxes.
In other words: If you are comfortable OK'ing a logistic regression model (because you looked at the coefficients and they made sense), you should be comfortable OK'ing a complex NN model (because the evaluation and interpretability modeling makes sense).
Nitpicking, but significant: Most models don't output decisions, they output predictions. Decision scientists then build a policy on top of the model. Key issue here is that the policy makers don't trust the predictions. But I posit they have no reason to trust the predictions of a logistic regression model any more than the predictions of a complex black box. Provided, of course, you deliver interpretability UX, confidence estimates, and strong statistical guarantees and tests. Which is possible for even the blackest of boxes.
If automatic justification is impossible for computers/black boxes, I believe it is impossible for humans too (as per Church-Turing). But let's say it is impossible. Do you think Google would use a white box model to optimize Adsense, because they can't interpret powerful deep learning solutions (like risk management for BlackRock: a very critical part of their business)?
I'd say Google came pretty close with https://distill.pub/2018/building-blocks/ (they are not the only players in the interpretability field, and plenty of methods are becoming available, in large part driven by academia not business: interpretability and fairness are not too important for the bottom line).
