An LLM in isolation is not a general purpose system, but with ChatGPT at least, most of the time you don't need to ask. In fact, it's increasingly difficult to force it to do "manual" maths, as it's strongly predisposed to do things like write and evaluate Python code instead of doing it "manually".

E.g. I verified just now and asked it to multiply two huge numbers, and it immediately spat out Python code, then evaluated it and gave me the result, rather than try to add the numbers itself.

It still needs to be trained on what calculators are, on how to use them, and on when it is appropriate to do so. Again, same as humans.