"I think this is semantics"
Of course; semantics refers to meaning of words; obviously disagreement between two parties will be about the meaning of words. Probably, you mean "polemics."
Physicists are professional doctors of philosophy; by tradition; they own the right to define any theoretical subject and they defined physics as the fundamental science defining nature; which is not correct. Physics is just one way of looking at nature.