And the same can be said of non-ML IT ! You always contrast better when you understand the whole history behind why you write something a certain way, even if you could just learn on the job seeing it over and over. It's like how they teach proper sorting by giving you all the bad ways first.

Also, it's not often but you do have to show creativity at times, to solve a new problem or something, and having an intuitive theoretical understanding goes a long way vs someone who learned via base mimicry.

I think instead of gatekeeping we could build bridges: be very clear that salary / responsibilities will be lower at first and judge on results. If an ML person is brilliant, he won't be threatened by an idiot Java dev. And if a Java dev is able to produce good results even if the way he reached them is less graceful, then an ML engineer should probably start shifting the second gear :D

Sure, I've been doing that "intuitions about equations" thing since 1993 (my undergrad thesis was on using gradient descent to train the weights of a dynamic programming algorithm that found e.coli gene). I generally agree, to be a top ML researcher, you need those skills in excess of the average (I worked with quite a few of those people at Google). To do state of the art work? Mostly hard work, lucky guesses, lots of compute power, and a huge support apparatus to make rapid experimentation easier.

But the vast majority of people working in ML don't need that. Sadly, most of the work I did for one of the world's most powerful machine learning systems was literally computing frequencies and then sorting by the frequency, so features that were more common were encoded in smaller varints, saving lots of disk space.