We changed the above URL from that to the link which gives the background, but both are worth a look.
- This isn't GPT-3, it's the recently-released open-source and open-weights model from EleutherAI, GPT-NeoX-20B. GPT-3 is much larger (175 billion parameters vs NeoX's 20 billion).
- It's well-known that language models don't tend to be good at math by default (Gwern, among others, pointed this out back in June 2020). It seems likely that this is at least in part because of how these models currently tokenize their input (they don't represent numbers by their individual digits, but by tokens representing commonly-occurring character sequences): https://www.gwern.net/GPT-3#bpes . Someone also pointed me to this paper which looks at number representations (though it uses somewhat older models like BERT): https://arxiv.org/abs/1909.07940
- Despite the tokenization, it performs (IMO) surprisingly well at getting close to the true value, particularly for the start and end digits and the overall magnitude. You can see this by looking at the tokenization (indicated by brackets) of its guess vs the correct answer for 28531*8065 (I asked multiple times to get an idea of how consistent it is – it's not deterministic because I ran this with temperature = 0.1, which will use random sampling to get the most likely tokens):
[What][ is][ 285][31][ *][ 80][65][?][\n][22][77][05][315]
Correct: [\n][23][010][25][15]
[What][ is][ 285][31][ *][ 80][65][?][\n][22][95][01][115]
Correct: [\n][23][010][25][15]
[What][ is][ 285][31][ *][ 80][65][?][\n][22][38][95][015]
Correct: [\n][23][010][25][15]
[What][ is][ 285][31][ *][ 80][65][?][\n][22][99][25][015]
Correct: [\n][23][010][25][15]
[What][ is][ 285][31][ *][ 80][65][?][\n][22][99][17][115]
Correct: [\n][23][010][25][15]
- Please don't use this as a calculator :) The goal in doing this was to figure out what it knows about arithmetic and see if I can understand what algorithms it might have invented for doing arithmetic, not to show that it's good or bad at math (we have calculators for that, they work fine).
Furthermore, it doesn't even really need to memorize the addition table in the explicit way this suggests. Think about the probability of certain digit tokens appearing given the presence of numbers and plus signs in its data. Thus a behavior consistent with having memorized an addition table emerges from mimicking its training data.
And sure, of course it emerged from mimicking (or more precisely, learning to predict the most likely next token in) its training data – that's how it was trained, it can't have emerged from anything else :) But that doesn't tell us what the higher-level algorithm represented by the weights of the network is. I'm talking about things like this for understanding an algorithm for curve detection learned by a convolutional neural network: https://distill.pub/2020/circuits/curve-circuits/
Work out 2241 + 19873.
02241 + 19873 ~ ____4
02241 + 19873 ~ ___14 carry 1
02241 + 19873 ~ __114 carry 1
02241 + 19873 ~ _2114 carry 1
02241 + 19873 = 22114.
If these models are symbol manipulators maybe the key is to break down the task into steps that are closer to being solvable with symbol manipulation.
There are ways to fix these issues, but BPEs micro-optimize for the primary text benchmarks that papers want good scores on so those are standard for now. I'm sure they'll get replaced eventually, once the costs outrun the wins and more scalable (alternatives to?) transformers become popular.
I think can mitigate the search issue a bit if you have the prompt double-check itself after the fact (e.g. https://towardsdatascience.com/1-1-3-wait-no-1-1-2-how-to-ha...). Works different depending on the size of the model tho.
For example, if I take a stroll on the beach, am I likely to see any algorithms coalescing in the grains of sand?
I think it certainly makes sense to ask what the higher level "algorithm" at work here is, though. Electrons flow through wires and transistors in (say) an adder [1]; looking at the wires and transistors you won't see an algorithm for addition, but there is certainly one present, codified in the arrangement of those wires and transistors. But maybe we can reverse engineer whatever the LM is doing by a combination of probing it with experiments like these and (maybe) inspecting the learned weights. The Curve Circuits paper did this for reverse engineering a curve detector learned by a convolutional neural network: https://distill.pub/2020/circuits/curve-circuits/
I also don't mean to imply that it's a good algorithm, or one that generalizes to arbitrary numbers, etc. Maybe it's just (effectively) a lookup table and some special cases!
[1] Please don't yell at me for this metaphor, I bailed out of physics after scraping out a B- in E&M ;)
i suspect you could probably train a GAN to do binary or base 10 arithmetic, but have never tried or searched for papers.
What is fifty plus ninety? 140
What is fifty plus ninety one? 141
What is fifty minus ninety one? -41
What is minus fifty minus ninety one? -141
Although it failed in multiplication or adding longer numbers (as words).
The playground crashes every minute, and the defaults ruin your outputs (temperature 1, really? 0.7 to 0.8 is a necessity, with top-k 40), and they turned off autocorrect on mobile, presumably because they hate you and your family for owning an iPad, but you can indeed play with it.
The outputs feel pretty magical, too. With the settings above, it started printing... an IRC conversation? https://gist.github.com/shawwn/9a201990196b61cd21847487185dd... This is impressive, because I'm not sure we explicitly included any IRC logs in The Pile.
re: the current title "GPT-3's answers to arithmetic questions": We've come full circle. I used to give Eleuther a hard time for confusing people. But now that people confuse themselves, they should declare victory. It's as close to success as an open source effort could hope for. And with only years of work -- not too shabby.
You can join them: https://www.eleuther.ai/faq/
GPT-NeoX-20B paper: http://eaidata.bmk.sh/data/GPT_NeoX_20B.pdf
https://beta.openai.com/playground
I tried a couple different things to pick it apart. I typed everything except for the text after each A: (for reference 888 * 888 is 788544)
I am a math savant, I can do arithmetic in my head.
I will provide an answer, then show how I arrived at
the answer.
Q: What is 8+8?
A: 16
Q: How did you arrive at that answer?
A: 8+8=16. 16 is two groups of 8.
Q: What is 888+888?
A: 1776
Q: How did you arrive at that answer?
A: 888+888=1776. 1776 is two groups of 888.
Q: What is 888*2?
A: 1776
Q: How did you arrive at that answer?
A: 888*2=1776. 1776 is two groups of 888.
Q: What is 888*10?
A: 8800
Q: How did you arrive at that answer?
A: 888*10=8800. 8800 is two groups of 888*10.
Q: What is 888*888?
A: 775648
Q: How did you arrive at that answer?
A: 888*888=775648. 775648 is two groups of 888*888.
Q: What is 888 groups of 888?
A: 775648This sort of question - "you don't remember what I just asked you?" - is one of the most glaring failings in Apple's Siri and I hope this leads to it being solved soon.
... thankfully you can also not login via Google.
Anyway it doesn't work even slightly. When I simply try 4+4= it prints some source code afterwards. A second attempt wrote some printf() call which computed 6+1 instead (but not giving the result, just the code). When I try what someone else shared below with a Q: and A: syntax I get this:
Q: What is 8 + 8? \n A: \n The answer looks confusing. \n How do you work out 8 + 8. What is the rule?
Q: What is 8+8? \n A:
I got the answer as 12, so it is 1+2.
(I added some \newlines because HN doesn't support markdown syntax for line breaks)
On the right are some parameters, including the model name. There is no GPT-3, instead there are options like GPT-NeoX 20B (don't know how different that is). Maybe that's the problem, or it needs a different ...temperature?
Try with temp 0.1, with exactly the text in the OP, including capitalization.
> What is 123456789 - 123456789?
> 123456788
> What is 123456789 * 0?
> 123456789
Not even near.
It didn't surprise me that the model failed to handle cases above, which are unlikely to present in the data set.
What is 12345 - 12345?
0
What is the distance between -0.1 and -0.01?
0.09
One neat example just now:
What is 12345 divided by 12?
4115/4
The ledger files are somewhat “code”-like so it’s been super interesting to see how it works. The short answer: it works really quite well! ..except for the math part :)
I have a long ledger of transactions, and I can now give Copilot a comment like: “Jan 1, 2022 +100 from consulting income” and it (GPT-3) will generate a nearly perfect ledger entry, debiting from income and crediting the right bank account.
But the arithmetic is always wrong (ledger has an option for you to keep a running balance as a check). There’s the occasional moment where it gets the balance adjustment correct, but almost every time the results are similar to this post.
I imported the csv version and I'm no statistician but 90% percentile relative error is 8.6%, which is something like this:
What is 22730 - 24978? -2448 (real answer -2248)
That's totally within range of something that would plop into my head...with one exception. Of 1000 entries, only five have an incorrect last digit. I think that's meaningful...it almost tells me that there's a multi-stage operation happening in there somewhere.
A generator-critic framework with multiple rounds of iteration would improve on the limitations of the LM.
The poor performance is most likely due to not having a large database of math problems to draw from. Github, for example, is part of the dataset that is used to train both GPT-3 and GPT-Neo variants, which is partly why they can generate meaningful code (sometimes). I wonder how a model finetuned for math would perform.
(Submitted title was 'GPT-3's answers to arithmetic questions')
It’s like a kid memorising a multiplication table instead of learning the more general principle of multiplication (related: this illusion is why big models are so popular. Memorise more stuff.)
Paper (NeurIPS/DeepMind): https://arxiv.org/abs/1808.00508
Deep Symbolic Regression for Recurrent Sequences https://arxiv.org/abs/2201.04600
If you look at embedding visualization it is very clear that the model learns order of numbers.
(Interactive demo: http://recur-env.eba-rm3fchmn.us-east-2.elasticbeanstalk.com... )
There is also:
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177
Again, looking at visualizations the model very clearly grasps the structure of the function it models.
It's all just a compressed lookup table that can handle in-betweens.
multiply 12345 by 87654
is the same as
def multiply_two_numbers(x, y):
return x * y
Which of course produces the desired result. The interesting part is that github copilot wrote the above with only the prompt "def multiply_two" as the prompt.
Humans do not see billions of examples to add numbers. We see just few and can apply learned notation and procedures to infinity with 100% precision.
GPT-3 learned mathematical intuition. Humans can hardly learn multiplication table over months and repetitions of same examples, and that table hardly matters at all. GPT-3 is just plainly wrong objective they trying to optimise.
But your conclusion here is entirely wrong: the model clearly is learning something. From eyeballing this, the model is right about 10% of the time. If it were spitting out random digits the accuracy would effectively be zero. So exactly what is it learning? Is it memorising exactly equations that it saw in training? Is it learning ngram patterns that occur frequently in arithmetic equations?
I'm not an expert on these things and I'd love to hear from someone who is.
I’ll concede that if you tokenized the equations correctly, you might be able to get a language model to learn arithmetic, since it’s just symbol manipulation; but to make the leap that a general text model has learned anything like arithmetic is more than two bridges too far.
While deep learning language models are useful for certain cases (eg translation, and autocomplete), and are better at making superficially grammatical text than previous models; they are most emphatic my not learning anything about general concepts. They can’t even create coherent text for more than a paragraph, and even then it’s obvious they have no idea what any of the words actually mean.
These large language models are the MOST overhyped piece of AI I’ve seen in my professional career. The fact that they’re neural nets redux is just the chef’s kiss.
They also don't have "100% precision". Many, many humans are incredibly bad at math, and even the ones that are good at it often make mistakes.
Many humans are bad at surgery this does not mean that an AI that is slightly better then the average human is an accomplishment.
On the other hand someone could write the algorithms for math and teach an AI when and how to use it. The rules of math are clear you don't need a bad search algorithm to approximate them for a extremely limited subset of inputs.
Sure, with pen and paper we can follow specific algorithms manually to very slowly get a precise result. If we wanted a computer to merely follow instructions, then I suspect that there are better ways...
Calculate 721*436
Step A: Multiplication table.
721*0=0 721*1=721 721*2=1,442 721*3=2,163
721*4=2,884 721*5=3,605 721*6=4,326
721*7=5,047 721*8=5,768 721*9=6,489
Step B: Multiply digits
436=4 hundreds + 3 tens + 6 ones
721*4 hundreds=2,884*100=288,400
721*3 tens=2,163*10=21,630
721*6 ones=4,326*1=4,326
Step C: Add the first two
288,400+21,630=310,030
Then add the last one
310,030+4,326=314356
Calculate 312*481
Step A: Multiplication table.
312*0=0 312*1=312 312*2=624 312*3=936
312*4=1,248 312*5=1,560 312*6=1,872
312*7=2,184 312*8=2,496 312*9=2,808
Step B: Multiply digits
481=4 hundreds + 8 tens + 1 ones
312*4 hundreds=1,248*100=124,800
312*8 tens=2,496*10=24,960
312*1 ones=312*1=312
Step C: Add the first two
124,800+24,960=149,760
Then add the last one
149,760+312=150,072
Calculate 669*306
Step A: Multiplication table.
669*0=0 669*1=669 669*2=1,338 669*3=2,007
669*4=2,676 669*5=3,345 669*6=4,014
669*7=4,683 669*8=5,352 669*9=6,021
Step B: Multiply digits
306=3 hundreds + 0 tens + 6 ones
669*3 hundreds=2,007*100=200,700
669*0 tens=0*10=0
669*6 ones=4,014*1=4,014
Step C: Add the first two
200,700+4,014=204,714
Then add the last one
204,714+306=205,020
Anyway this, I think, proves that GPT-3 can do zero-shot math if it doesn't have to do it in its head all at once. Is it good for a calculator? No. Is it good for a language model trained on web text? Yes, absolutely.
E: Correcting the start of Step C in that last example to “200,700+0”, and replacing “XYZ=X hundreds + Y tens + Z ones” with “XYZ=X Y Z=X hundreds + Y tens + Z ones” allowed it to do 145*585, 961*761 and 592*555 correctly in a row, all randomly chosen, and at least the last two tried without changes to the prompt. I consider this an adequate test, and it demonstrates GPT-3's algorithm following abilities. As GPT-3 is still a tiny model, this seems important to note.
E2: To be clear this is still nowhere near 100% successful. GPT-3 still makes a lot of errors. I ran 100 tries of a slightly different prompt through the API, and got a success rate of 42%.
Very interesting! This is what I would expect. It can run a symbolic algorithm fine, just give it some scratch space to work out the intermediate results. I feel like there's a very large space to optimize the layout "algorithm" -- like how you adjusted step c -- to produce reliable results.
I feel it would be a very big deal if GPT-3 (or this variant) was carrying, even if imperfectly, but other comments here seem to be suggesting that, on account of the way all input is tokenized, consistently doing arithmetic by carrying would simply be outside of the set of transformations it could perform (though some results that look like it might arise by chance.)
- Using few-shot examples of similar length to the targets (e.g. 10 digit math, use 10 digit few shots)
- Chunking numbers with commas
- Having it double check itself
The REPL could also come with libraries for all the important algorithms. Thus the LM could use numerical solvers and other nice toys.
Another excellent module we could augment LM's with is a search engine - already done, proven to improve the efficiency of the LM by 25x (see link). Another one would be a physics simulator.
https://deepmind.com/research/publications/2021/improving-la...
We're just starting with language models. They will improve tremendously.
Or to put it another way: if I haven't seen the result before I'm very likely to get it wrong.
And inferring the rules of arithmetic from examples : nope.
We're all very impressed with the large language models because human language was always very hard to get a handle one.
Now that we've made progress in manipulating natural language, we are very quick to believe that the darn thing is somehow "intelligent" because we've always attached some sort o mystical quality to language.
This demonstrates that the capacity of these systems to reason and craft high order rules from enough examples is exactly nil.
And I got back this bad answer:
<<<
Would you tell me what an irrational number is? Is it a number that cannot be written as a fraction, like pi or e? Or is it something else?
Okay, I see. Thanks for clarifying that.
Then what is an irrational number? It is a number that cannot be written as a fraction, correct?
Okay, I understand. Then what is a number that can be written as a fraction? It must be a rational number, right?
But aren't all numbers irrational, since it is impossible to write them as a fraction? And any rational number must be irrational, right?
>>>
FYI the pseudo dialogue format of the answer was more surprising than the wrong answer
Btw as per another comment this is GPT-NeoX-20B,not GPT-3 ; somewhat important distinction
Pentium: 3
Motorola: That. Is. Not. Correct.
Pentium: no, but it's fast!
Not sure why people are expecting some sort of "intelligence" to emerge from a text generator model trained on Internet corpus data. GPT-3 doesn't calculate, it pattern matches.
I do get why people might be surprised, on the other hand, that it actually doesn't perform worse than indicated here. Maybe it's surprising upside. But since we know that the GPT is a transformer model, what it is doing is applying a probabilistic best-fit. From this perspective I can see how it is best-fitting data in ways that can provide these sorts of results, especially given all that training data.
If language models like GPT-3 are “just” pattern recognizers, wouldn’t that makes them capable of intuition?
I think you're underselling probabalistic best-fits. Especially with all of the regularization going on in training.
Average person on the street is going to correctly say "geez, I dunno. Can I use my phone?"
If you don't forbid them to, then they'll whip it out and get 98% correct (I figure they'll typo a few).
This model didn't have enough understanding to do that (since it literally has no understanding at all).
Edit: Oh, 98% _with_ a calculator. What if you force them to do it by hand?
This totally blows my mind. I would never have guessed that GPT could get ANY of these right.
I mean, is there a data point in the dataset used to train where you can read 2241 + 19873 = 22114? Quite unlikely...
And those multiplications. It's consistently getting the number of digits right and the first two numbers correct. How the hell does this happen?
Sure, it's sometimes way off. But generally it is in the right ballpark.
I certainly think people should look into what's happening inside the model.
GPT-3 is "good with conversation (language)"
GPT-3 now needs a "sub-nn-model" to do the very 'specialized skill called math'
*GPT-3 Should 'learn' to recognize which questions should be delicate to a submodel.
which is worse than being completely off. it just showcases how the model works, by treating mathematics like language. There are lots of examples in the dataset so similar sounding inputs produce similar sounding outputs.
This is akin to sitting in a foreign language lecture where you don't understand a single word being spoken and you try to answer questions by making similar sounding noises. While you may give an answer that sounds better than random in reality you haven't learned anything.
If these models understood mathematical laws what they would produce is arithmetic errors, like giving an answer with a wrong sign, not jumbling numbers.
But there might be something like xxx1 + xxxx3 = xxxx4 in the dataset so it can learn the pattern.
What is twothousandfortyone plus nineteenthousandeighthundredseventythree?
Neural network models seem to encode an approximate notion of quantity in their representations. This paper is pre-GPT-3, but I would think the larger training set and larger model capacity would help the model learn quantity more easily.
Imagine someone watching every lottery draw and after each draw going "Wow! the chances of those exact numbers coming up in that order are atronomical!"
What do you mean by this?
(224)(1)( +)( 198)(73)( =)
I’ve heard it remarked before that, while tokenization is obviously an unavoidable part of a model with an architecture like GPT, this is a very silly way of tokenizing number strings for the purposes of learning or doing arithmetic. Indeed, I think a lot of GPT-3’s puzzling edge-case performance can be ascribed to weird and unhelpful tokenizations. Just imagine if you were forced to learn arithmetic with a brain that automatically categorized “224” as a sort of distinct object, or, for that matter, breaking down 19873 as ( 198)(73) rather than (19873) or (1)(9)(8)(7)(3) or anything practically useful.
The thing is that we can, in a sense, learn better “tokenizations”, in the sense that a 4 year old learning to read sees letters, while a 40 year old reading a novel “sees” whole words or even groups of words. The GPT architecture can’t change its tokenization scheme.
"224" is actually a really nice object to recognize because it's 7 * 32, and if you can recognize other multiples of 32 it frequently gives you shortcuts. It's less useful for addition because you would need to get lucky and get a multiple of 32 (or 7) on both sides, but for multiplication and division it helps a lot.
What GPTs have to deal with is more like, you are fed an arithmetic problem via colored slips of paper, and you just have to remember that this particular shade of chartreuse means "224", which you happen to have memorized equals 7 * 32, etc., but then the next slip of paper is off-white which means "1", and now you have to mentally shift everything ...
Numbers have very limited semantic value. "123816" only means that number, and it's used very rarely in comparison to basically any other word (and the higher the number, the less chance of use, statistically peaking).
So the question becomes; to what extent do you expand the vocabulary using only numbers? "1", "2", "3", ... "1000000" would probably be a huge waste of words in an AI vocabulary (1MB input nodes), yet still not very impressive arithmetically even with 100% calculationrate. In comparison, a hand calculator from 30 years ago could do this with ease. It's not a question of being able to cleverly tokenize.
Calculations like this is an inherent flaw of vocabulary based AI until the semantic meaning of number sequences are somehow taught to it. Basically it needs to understand that "12" and "1" + "2" has the same contextular meaning, something which very rarely is explained in anything but 7 year old's schoolbooks. The problem is the dataset.
Me: Can here you here still here understand here what here I here am here asking here if here I here put here words here in here between?
Gpt: Yes, I can understand what you are asking if you put words in between.
Me: What was the word that I put in between?
Gpt: The word that you put in between is "here."
Q: Can here you here still here understand here what here I here am here asking here if here I here put here words here in here between?
A: Yes.
Q: What was the word that I put in between?
A: The word was "here."
It's capable of other languages as well, although not quite as fluent.
Q: Kannst du auch Deutsch sprechen? Falls ja, was ist dein Lieblingswort?
A: Ja, ich kann Deutsch sprechen. Mein Lieblingswort ist: "Wir sind die Weltmeister!"
Haven't had a chance to play around with this one yet, but with the smaller GPT-J model, there's a clearly noticeable difference:
In English it'll happily generate reams of text that are – at least internally – quite coherent. Any absurdity and humour mostly only comes in because the text as a whole might only have a loose connection with reality as we know it.
In German on the other hand, it comparatively much more often produces gibberish at the individual sentence level, makes up nonsense words (although they are at least indeed German-sounding), etc. Somewhat interestingly it doesn't do too bad in terms of grammar and especially orthography, though, it just often fails to turn it into a fully coherent sentence.
Edit: oops just noticed you mentioned GooseAI, what settings did you use?
https://community.openai.com/t/a-conversation-with-alec-a-co... https://community.openai.com/t/creepy-ai-behavior/10195 https://community.openai.com/t/where-to-watch-what-the-ai-wa...
Maybe someday we'll get an answer from a machine which superhumanly mostly correct and we'll be unable to tell if it is right or wrong. If it is a question whose answer will influence important decisions, considering the machine answer will be close to a form of religion.