And then we can drill down even further where everything is just physics with atoms, quantum mechanics, etc.

So you're not different from a computer. Both are just physics.

But that's not a useful world view in my opinion.

I think "regular" and "non-regular" programming is a useful distinction.

In regular programming, I have to write explicit implementations of the algorithms in the program.

In "non-regular programming" (neural networks), I just have to know how to set up and train neural networks.

Once I do that, the neural networks can be trained to evolve algorithms that I myself don't know how to implement.

Don't you see the big difference between "I have to code the algorithms" and "the computer does it for me"?