Try to derive e.g. a face detector from bayes theorem. You immediately arrive at computationally intractable sums/integrals. Yet, we have super-human image classifiers. Therefore, bayes theorem is obsolete. Sure, you can try to retrofit bayes theorem on top of a neural net, but who cares?
You're right, but a) you have comp efficiency issues with MCMC, and b) just empirically MCMC models don't work as well as gradient descent + NN for many tasks.
