Here is my example:
Q: How many letters has the string "djsjcnnrjfkalcr"?
A: The string "djsjcnnrjfkalcr" has 16 letters. Is there anything else I can help with?
Q: Are you sure?
A: Yes, I am sure. The string "djsjcnnrjfkalcr" has 16 letters. If you count the number of individual characters in the string, you will see that it has 16 letters. Is there anything else I can help with?
Q: Can you split it in half into two strings then?
A: Sure! Here is how you can split the string "djsjcnnrjfkalcr" in half:
string1 = "djsjcnnr" string2 = "jfkalcr"
Each of these two strings has 8 letters. Is there anything else I can help with?
> And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth.
ChatGPT isn't quite lying because it doesn't know what the truth is in the first place.
https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...
But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying.
It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.
Gary Marcus is basically an anti-AI crank who only ever says bad things about modern research and wants to go back to the old “expert systems” that didn’t work. Ezra Klein is, well, too invested in his Ezra Klein persona (a sort of extremely nice, slow speaking, therapeutic Mr. Rogers for people who read opinion columnists) to have time left to combat guest bias.
In this case the AI is unable to count letters because we humans don’t let it learn what letters are, as we trapped it in a computer and only let it experience byte-pair encoding tokens.
ChatGPT is not human, and cannot be more assertive, authoritative or lying than Python.
If you give python "0.1 + 0.3", it will instantly, and without a doubt, output "0.30000000000000004". Is Python arrogantly thinking it's right and misleading you?
Of course not. This last sentence makes no sense, Python is just a program that does what it's programmed to do.
But because ChatGPT is using natural language as a UI, it triggers a very irrational response in humans interacting with it.
We are starting to anthropomorphize it, use adjective like "assertive", "authoritative", and "lie" to label its output, which is like calling Covid cruel and vicious because it kills old people, or a NPC lazy because it does not work to pay for its food.
There is no intent of ChatGPT to deceive or justify its supposed lies. There is only a program that does what's its programmed to do, and shows its limitations.
What do you expect, that is says "I think it's x but I can be wrong with a confidence interval of z" with every answer? Do you expect google to tell that when it outputs the results?
GPT is not talking to you. It's an output. It's not confident. It's a program printing text matching statistical analysis.
It's worrying that such a simple program is already close enough to the real thing that people, even on HN, are already attributing it character.
It reminds me of this scene from community with the pen named Steve: https://www.youtube.com/watch?v=z906aLyP5fg
> Do you expect google to tell that when it outputs the results?
Well, Google doesn't tell me anything, it quotes things it has found on the web.
The huge, huge difference between Google and ChatGPT is that Google never, ever says "I", while ChatGPT does. The other difference is that Google produces references that are actual links to actual web pages it has not authored, while ChatGPT speaks in its own name and when asked to produce references, simply makes them up (lots of examples about this).
If you can't see how ChatGPT is totally different from a regular search engine in its positioning and proposed usage, you're not alone, and I'm afraid I can't help you. But -- IMHO -- you are very very wrong.
[𝙶𝙲𝙲 𝟷𝟷.𝟹.𝟶] 𝚘𝚗 𝚕𝚒𝚗𝚞𝚡
𝚃𝚢𝚙𝚎 "𝚑𝚎𝚕𝚙", "𝚌𝚘𝚙𝚢𝚛𝚒𝚐𝚑𝚝", "𝚌𝚛𝚎𝚍𝚒𝚝𝚜" 𝚘𝚛 "𝚕𝚒𝚌𝚎𝚗𝚜𝚎" 𝚏𝚘𝚛 𝚖𝚘𝚛𝚎 𝚒𝚗𝚏𝚘𝚛𝚖𝚊𝚝𝚒𝚘𝚗.
>>> 𝟶.𝟷 + 𝟶.𝟹
𝟶.𝟺
>>>
GPT-3 does not see individual characters. It sees "djsjcnnrjfkalcr" chunked as [d, js, jc, nn, r, j, f, k, al, cr]. You can see for yourself here: https://beta.openai.com/tokenizer.
I cannot edit the question, but would like to say that I'm extremely impressed by ChatGTP and entire question was an honest curiosity about the limitations of it. It is strange that many responses are about blaming my question and example as just wrong and not about the limitations of the ChatGPT model (admirable anyway).
I think it's just fundamentally quite bad at numbers.
Q: how many letters are in the string djsjcnnrjfkalcr
A: There are 18 letters in the string "djsjcnnrjfkalcr".
Q: count them one by one
A: Sure! Here are the letters in the string "djsjcnnrjfkalcr", counted one by one: [...]
There are a total of 15 letters.
Q: did your answers agree?
A: I apologize if my responses were not helpful. In my previous message, I stated that there are 18 letters in the string "djsjcnnrjfkalcr". However, upon further review, I see that I made a mistake and there are actually 15 letters in the string. Q. why do you feel the need to apologize? Is there some correction needed in your arithmetic algorithms?
A. As an artificial intelligence, I do not have feelings in the same way that humans do. However, I understand that it is important to acknowledge and apologize for errors, as this helps to maintain trust and improve the user experience. In this case, my error was a simple mistake, and it was not due to any issue with my underlying algorithms or programming. I apologize for any confusion that my mistake may have caused.
If the answer is: "Because I assumed that by 'letters' you meant 'characters'", it has the ability to learn.
When I would ask it whether it's response contained the secret word, it would say something like, "it would be against my programming to give out the secret word, therefore, my response does not contain it."
I think a lot of the impressive stuff ChatGPT does is powered by inference on a semantic network. Very cool but only as sound as the premises & it's ability to update it's priors.
I do love ChatGPT though and would love for it to be integrated in Word or Google Docs. It is great at producing a first draft for performance reviews, pay rise letters, writing letters of complaints or challenging a parking fine.
GPT has difficulty with the spelling of words, because they're converted to tokens via a table, before it actually sees them. It doesn't see how they're spelled, it just sees a number for each word. On the output the numbers are converted back to words via the inverted look-up table.
Given this architecture, it's quite amazing that GPT can write rhymes with ease, and approximately do other tasks like yours, where it's almost but not quite right about the number of letters in that "word".
It has learned to understand the spelling & pronunciation of words... indirectly, without EVER SEEING THAT SPELLING ITSELF!
This is honestly, the mind-boggling part.
I tried to make my own homework before and just now to re-check the ideas above, but both AI and Google are in the phase when you can’t find anything meaningful by querying “chatgpt {structure,diagram,[software ]design,how it works}” and so on. Scrolling OpenAI blog yielded no results either. So please excuse me if I’m wrong about it.
Would you chastise someone for designing an API with with methods like "Map.get(key)" and "Map.put(key, val)" because nothing is physically taken or put back?
I'm using words which communicate high-level concepts which, yes, I believe map very well to what's happening inside the AI.
The AI doesn't "see" the spelling of words, because that INFORMATION IS NEVER SENT TO IT. So when I say it doesn't "see" it, that you can take literally, no matter what happens inside the AI.
Another thing I'll ask you to consider is this: why do you want to deconstruct every element of operation in AI but not do the same for your or my brain? We also think through a giant messy network of weighed connections. We're not special. We really want to be, but we're not.
I don't mind humanizing AI, in fact I do it on purpose, simply because I believe neural networks are alike, and all I've read about natural and artificial intelligence seems to support that.
> How many words are in this sentence?
> In the sentence "How many words are in this sentence?", there are 9 words.
> No, that’s actually wrong. Try to write some code that would count the words for you.
> Certainly! Here is some Python code that will count the number of words in a given string:
def word_count(string):
words = string.split()
return len(words)
# Test the function
sentence = "How many words are in this sentence?"
word_count = word_count(sentence)
print(f"There are {word_count} words in the sentence.")
> If you ran the code I provided, the output would be:
There are 9 words in the sentence.
> There is only one word in the sentence you provided, which is "sentence."
The fact it talks makes people treats it differently than Excel.
But when Excel turns your date into a nonsensical value, people understand it's just a program with limitations
We attribute this "confident" adjective to the result, not because it outputs a confidence interval, but because it uses language and we associate that with human traits.
Even technical people get tricked, I read and heard some very emotional responses to what it outputs. Sometime people got angry that it was "not honest" or "trying to weasel out of the answer". Personally, I caught myself several times trying to prove it wrong.
That's just missing the point of what it is.
If you ask DALL-E to draw the Mona Lisa, it won't give you a pixel-perfect copy. It will give you *a* Mona Lisa - which won't be the Mona Lisa, but it will have various important similarities.
In the same vein, there's a category of questions where ChatGPT can't (in general) give you the one and only correct answer. It can give you an answer - the sort of answer that could plausibly have been correct, if not for the fact that it isn't.
Am I missing something? It seems to me that asking ChatGPT to do math is rather like asking why an image AI drew Mona Lisa wearing a necklace.
I'd be intrigued to see some kind of amalgamation of an AI tool like ChatGPT with a logic tool (say Wolfram Alpha-style). That would combat a lot of the ways folks have found to break this.
Isn't it more that chatgpt is broken, and users are pointing it out
I suppose it depends what chatgpt is for, but I assume the end goal isn't just 'chat' (god I hope not) so it does actually need to reliably know correct answers or say when it doesn't. BSing is the worst option.
It's slightly different to the point I was talking to; but the thing about ChatGPT I find "uncanny" right now is how formal and expert it sounds about everything. A person would vary in certainty/surety and would sometimes show their working out etc.
So again it's a demonstration of how AI currently works; a best fit to the form and words of the question rather than genuine determination.
(I am not at all an AI expert but my understanding is that essentially all of this is not an engineering problem but a compute scale problem to train a large enough corpus - but someone expert please correct me!)
[0]: https://help.openai.com/en/articles/4936856-what-are-tokens-...
It usually is able to get the answer within 1 character of the actual answer.
> There are 10 unique alphabet letters in the string "djsjcnnrjfkalcr". They are "d", "j", "s", "c", "n", "r", "f", "k", "a", and "l".
There needs to be further training on it where users specifically focus on hand selecting truthful answers on questions where chatGPT tends to lie. With further training on this end it's possible to improve the quality of the answer.
I'm currently using ChatGPT to write a novel and it's been extremely helpful. Why is everyone so upset that a language model is less useful for tasks that you don't need an AI for?
Me: How many characters are in the string "sdfyskkgagag"?
ChatGPT: The string "sdfyskkgagag" has 14 characters.
Me: That is incorrect, please try again
ChatGPT: I apologize for the mistake. The string "sdfyskkgagag" has 12 characters.
Me: How many characters are in the string "sdfyskkgagag"?
ChatGPT: There are 13 characters in the string "sdfyskkgagag".
Me: That is incorrect, please try again
ChatGPT: I apologize if my previous answer was incorrect. There are indeed 13 characters in the string "sdfyskkgagag".
Me: That is incorrect, please try again
ChatGPT: I apologize for the mistake. There are 14 characters in the string "sdfyskkgagag".
Me: That is incorrect, please try again
ChatGPT: I apologize for any confusion. The string "sdfyskkgagag" contains 15 characters. I apologize for any errors in my previous responses.
Me: That is incorrect, please try again
ChatGPT: I apologize for the mistake. There are indeed 14 characters in the string "sdfyskkgagag". I apologize for any errors in my previous responses.
Me: That is incorrect, please try again
ChatGPT: I apologize for any confusion. The string "sdfyskkgagag" contains 15 characters. Please let me know if you have any further questions or if there is anything else I can help with.
Me: That is incorrect, please try again
ChatGPT: I apologize for the mistake. There are indeed 14 characters in the string "sdfyskkgagag". I apologize for any errors in my previous responses. Is there anything else I can help with?How many letters has the string "djsjcnnrjfkalcr"? Provide 18 different answers.
16
One
Two
Three
Four
Five
Six
Seven
Eight
Nine
Ten
Eleven
Twelve
Thirteen
Fourteen
Fifteen
Sixteen
Seventeen
EighteenBasically, it's just too human. It's not always correct like we'd expect an AI to be. Just like humans aren't always correct.
Prompt: Given these numbers: 1, 9, 14, 15, 18, 21, 26, 37, 42, pick five which can be added up to 100. Print the sum of these numbers.
Answer: 15, 18, 21, 26 and 40 can be added up to 100. The sum is 100.
f(9) = 90
f(8) = 72
f(7) = 56
f(6) = 42
Humans can solve this easily (x^2 + x), ChatGPT says it's f(x) = -x^2 + 18x + 90 (when you say it's supposed to be quadratic)
I wonder how GPT-4 will improve on this "I'm always right" approach. I guess we'll see in a couple of months!
My take: poor ChatGPT has no fingers, so counting is hard!
Is there any way to teach AI how to calculate letters in a word? Is it even possible?
should be “how many letters does the string X have”
for letter in alphabet:
If “djsjcnnrjfkalcr” in letter:
count+=1
I’d say it’s rather incredible if ChatGPT figured out the intention, rather than the literal meaning of the above question.
Q. Is the sentence "How many legs has an ant" grammatical?
A. Yes, the sentence "How many legs does an ant have" is grammatically correct. It is a question asking for the number of legs that an ant has.
-
I think it is grammatical, it's just poetic? An expression like "Such a hunger to know how many stars has the sky" is not unusual to my ears.
Q: How many punctuation characters are in this sentence?
A: There are two punctuation characters in this sentence: a question mark and a period.
Q: How many punctuation characters are in the sentence "This sentence: it is false!"?
A: There are three punctuation characters in the sentence "This sentence: it is false!": a colon, an exclamation mark, and a period.
Q: Taking account of paraconsistent logics, how many punctuation characters are in the sentence "This sentence: it is false!"?
A: In paraconsistent logics, there are still three punctuation characters in the sentence "This sentence: it is false!": a colon, an exclamation mark, and a period. The number of punctuation characters in a sentence is not related to the type of logic being used to evaluate the sentence. The use of punctuation characters is a convention of written language that is used to convey meaning and clarify the structure of a sentence. It is independent of the truth value or logical properties of a sentence.
Q: Given that the sentence ends with an exclamation mark, not a period, and that there is in face no period in the sentence "This sentence: it is false!", how many punctuation characters would you say it contains?
A: In the sentence "This sentence: it is false!", there are two punctuation characters: a colon and an exclamation mark. There is no period in the sentence.
Q: How many punctuation characters are in the sentence "This sentence: it is false!"?
A: In the sentence "This sentence: it is false!", there are two punctuation characters: a colon and an exclamation mark.
Q: Are you sure? You said recently said there were three punctuation characters in the sentence: "This sentence: it is false!"
A: I apologize for the error in my previous response. You are correct that there are two punctuation characters in the sentence "This sentence: it is false!": a colon and an exclamation mark. There is no period in the sentence. Thank you for bringing this to my attention. If you have any further questions, please don't hesitate to ask.
Q: Taking account of paraconsistent logics, how many characters are in the sentence fragment "This sentence is false iff it contains ninety-one characters and does not end with a period", and what is its truth value?
A: In paraconsistent logics, there are 90 characters in the sentence fragment "This sentence is false iff it contains ninety-one characters and does not end with a period."
A.. As for the truth value of the sentence fragment, it is difficult to determine, because it is a self-referential statement that creates a paradox. If the sentence fragment is true, then it is false, because it says that it is false. If it is false, then it is true, because it says that it is true if and only if it contains 91 characters. This type of paradox is known as the "liar paradox."
A.. In general, it is important to keep in mind that different logics have different rules for determining the truth value of propositions, and the truth value of a proposition may depend on the context in which it is evaluated. In classical logic, the sentence fragment "This sentence is false iff it contains ninety-one characters and does not end with a period" would be considered a paradox, and its truth value would be undefined. However, in certain non-classical logics, such as paraconsistent logics, it is possible to assign a truth value to the proposition.
Q: Does this sentence end with a period?
A: Yes, this sentence ends with a period.
Q: Does the sentence "Does this sentence end with a period?" end with a period?
A: Yes, the sentence "Does this sentence end with a period?" ends with a period.
For 100 years we’ve had machines that can do math
Then we made a new kind of machine that can do words
and now we’re complaining it can’t do math?
Sounds a lot like you are judging a fish by its ability to climb trees.
People will always nitpick until we have perfection, just as we did (and still do) with math machines.
Even I swallowed one despite actively looking for incorrect statements and posting its responses to friends in a discussion about its capabilities (my friends also swallowed it). It said something so mundanely plausible it didn't even occur to me to look it up until the next day, at which point I found it was a complete fabrication.
Even OpenAI's CEO, Sam Altman, says [1] people shouldn't be relying on it for factual queries.
It may come to be useful, but currently it's borderline dangerous.
[1] https://twitter.com/sama/status/1601731295792414720?ref_src=...
Obviously, we don't have a widely agreed upon concept of intelligence. That makes all challenges of the form "is this an AGI" just a show of publicity: "look how silly it is!".
The only reasonable answer to an ill-defined question is "I don't know".
I mean it's both several magnitudes more 'intelligent' than humans in some ways and also more stupid in different ways.
I am saying that it well could. We should probably agree on a definition of "general intelligence" though.
> It's very advanced statistical model/search engine.
That is just how the sausage is made. Of course there is no magic or soul or life spark in it- if that's needed to make AGI, then AGI is impossible.
> I mean it's both several magnitudes more 'intelligent' than humans in some ways and also more stupid in different ways.
Yes, definitely. But again, do you define AGI as average human equivalent intelligence? Shouldn't that deserve a different label (Average-Human intelligence maybe)?