Why don't LLMs ask for calculators? (opens in new tab)

But they only know about specific tools with few or zero shot methods, whereas modern humans are highly trained to specifically always route all math through a calculator, it's "in the fine tune" not "in the prompt".

It doesn't use up any "context" to remember that calculators exist.

With the partial exception of Python code, models know how to solve problems with code and even 1.5B models do amazing things if you prompt them with something like "Use the Z3 solver"

Basic math problems should be treated as a special vs general tool use. Why use Python when the computer evaluating the LLM can do math itself?

You still can't get 100% reliability that would be necessary for certain problem domains.

There are going to be some level of hallucination errors in the translation to the agent or code. If it is a complex problem, those will compound.

The author adressss this and argues that it misses the point:

> Now, some might interject here and say we could, of course, train the LLM to ask for a calculator. However, that would not make them intelligent. Humans require no training at all for calculators, as they are such intuitive instruments. We use them simply because we have understanding for the capability they provide.

So the real question behind the headline is why LLMs don't learn to ask for a calculator by themselves, if both the the definition of a calculator and the fact that LLMs are bad at math are part of the training data.

Yes, but the takeaway message is that even laypeople can invent how LLMs should work.

Especially for a problem as well-known as this, expect code for it to be seen by the model during training at least a thousand times in different languages, flavors etc. This, on its own, is nothing new.

That wasn't the point. The Math multiplication errors allow us to peer into the lack of reasoning across all domains.

Yes, you can add tools. But hallucinations will still be there. Tools allow you to cut down on the steps the LLM has to perform. However, if you have a complex problem with many steps, there will be translation errors at some point coordinating the tools.

Furthermore, if there is some other tool needed to get the result you need, the LLM isn't going to tell you. It will typically make up the result.

The point is that the LLM cannot figure out on its own that it needs to use the tool, you have to explicitly ask for it.

I think he means that you don't need to train a human to understand that a calculator is useful, and in particular when a problem is hard enough that you need to bust out a calculator. That sort of logic is self-apparent in humans, but struggles to be consistently evoked in LLMs.

That said, I was using simple +*-/ calculators as a small child and I don't think I needed to be taught anything other than MC/MR. The tool is intuitive if you are familiar with formal written arithmetic (of course hunter-gatherers couldn't make sense of it).

I remember having several days of lessons on how to use calculators in elementary, middle, and high school. They are "intuitive", but not all the functions of them are, and if you fully rely on intuition, you might expect them to do things like respect order of operations, which they very well might not.

As a kid of about 5 or 6 years old I used my first calculator with no instruction whatsoever. We are not talking about scientific calculators. Addition, Multiplication. It does not require training or instruction, just a minute of exploration.

The claim isn't "LLMs don't use tools", the author is saying that LLMs can't make reliable inferences regarding their own knowledge or capabilities which fundamentally limits their usefulness for many tasks. LLMs "know" that LLMs can't do math reliably, LLMs "know" that calculators can do math reliably, and yet LLMs generally just soldier on and try to do math themselves when asked. You can of course RL it or prompt it into writing javascript when it sees math but so far LLMs haven't been capable of generalizing the process of "I am bad at X" + "Thing is good at X" -> "I should ask for Thing to do X" unless that specific chain of thought is common in the training data.

The solution so far has just been to throw more RL or carefully crafted synthetic data at it but its arguably more pavlovian than it is generalized learning.

Someone could teach a dog to ring a bell that says "food" on it, and you could reasonably argue that it is using a tool. Will it then know to ring a bell that says "walk" when it wants to go outside?

The post is honestly very normal — many mid level intellectuals think this way and love these talk downs of LLMs. I don’t understand it for the life of me.

Two reasons:

1. LLMs "think" in terms of tokens, which usually are around 4 characters each. While humans only have to memorize 10x10 multiplication tables to perform multiplication of large numbers, LLMs have to memorize a 10000x10000 table, which is much more difficult.

2. LLMs can't "think in their head", so you have to make them spell out each step of the multiplication, just like (most) humans can't multiply huge numbers without intermediate steps.

A simple way to demonstrate this is to ask an LLM for the birth year of a celebrity and then whether that number is even or odd. The answer will be correct almost every time. But if you ask whether the birth year of a celebrity is even or odd and forbid spelling out the year, the accuracy will be barely above 50 %.

Are there a lot of examples written out of people talking through running that algorithm? I’d guess not.

If you don't give a human a calculator, they do the calculation in their head. The same with an LLM. If you give a human a calculator, they will use it. The same with an LLM. Both will say a calculator can do arithmetic better.

I don't believe current LLMs are AGIs but this article's argument is a poor one.

> Really though I think the author is highlighting that LLMs are not being used efficiently at the present moment.

Yes, that is a key point. It isn't to say they are useless tools, but that they aren't intelligent tools and that has significant meaning for what tasks we think they are appropriate for.

Unfortunately, nearly everyone has misinterpreted the intent as showing LLMs can't use tools. The point is about how LLMs work differently than most think that they do.

That was during the great plugin times. Haven't seen plugins for a while. Do they still exist?

The free version, at least once you are logged in, does as well. I don't pay OpenAI, and for the prompt

"Calculate the difference between the two biggest primes less than the factorial of 20"

it wrote the following code:

import sympy

# Calculate 20!

factorial_20 = sympy.factorial(20)

# Find the two largest primes less than 20!

largest_prime = sympy.prevprime(factorial_20)

second_largest_prime = sympy.prevprime(largest_prime)

# Calculate the difference

difference = largest_prime - second_largest_prime

difference

executed it, and produced the correct result, 40.

Yeah, but that's tool use. It's not the LLM realizing it needs a calculator, but rather a tool use prompt (instruction) causing it to predict a tool invocation sequence that is then intercepted by the model-calling scaffolding.

With reasoning models they could choose to do it slightly differently - just tell the model what tools are available, and how to invoke them, and let it decide when to use them in more flexible fashion, rather than rely on fine tuning to follow tool use instructions.