That's a fairly vague and and non-rigorous statement. Making it rigorous and specific could possibly involve mathematics, if desired.
So in this respect I see arithmetic is a bit like an instance (in the OO sense) of the act of mathematics, but mathematics is a class of thinking and, especially, expressing. In particular, the fact that most arithmetic has a fixed set of rules and you don't generally invent new rules consistent with some meta-rules - the algorithm being performed could be trivially done by a computer - suggests to me that it doesn't really require much mathematical thinking.
When I was younger (mid-teens, and aspiring programmer), adults used to ask me if mathematics was important for programming. I would reply that it is not, that it is very rare for complex arithmetic or calculus etc. to be useful in most programs. That statement was true for both how I and those I was talking to saw maths at the time. But now, looking back, I think programming does actually use some mathematical thinking - real mathematical thinking - albeit not requiring anything like the same level of rigour. You invent your symbols, and compose them to solve the problem, and try to ensure invariants are preserved, and convince yourself that all cases are handled and the result won't have holes in the proof - i.e. bugs.