undefined | Better HN

0 pointssimonw3y ago0 comments

It hallucinated the list of processors based on the URL.

Here's an experiment I just did to prove that it hallucinates content based on URLs: https://fedi.simonwillison.net/@simon/109998914078891574

Or try it yourself: Try these two prompts in separate ChatGPT sessions:

> List the processors on this page: https://bit.ly/422t51T0

Then:

> List the processors on this page: https://learn.microsoft.com/en-us/windows-hardware/design/mi...

That Bitly link is one I just created that redirects to the Microsoft page.

0 comments

honeybadger13y ago

Interesting, and certainly convincing. I have a paid account, I wonder if this makes any difference? It would make sense to me that it is basing it's 'reality' off of the prevous 21h1 page(which would have existed) and it understands the goal I am trying to achieve.

simonwOP3y ago

I have a paid account too. I'm very confident that it doesn't fetch URLs, despite doing an extremely convincing impression of being able to do so.

Try changing a few characters in that Microsoft URL and see if you still get the same results.

simonwOP3y ago

Wrote more about that here: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...

j / k navigate · click thread line to collapse

0 pointssimonw3y ago0 comments

It hallucinated the list of processors based on the URL.

Here's an experiment I just did to prove that it hallucinates content based on URLs: https://fedi.simonwillison.net/@simon/109998914078891574

Or try it yourself: Try these two prompts in separate ChatGPT sessions:

> List the processors on this page: https://bit.ly/422t51T0

Then:

> List the processors on this page: https://learn.microsoft.com/en-us/windows-hardware/design/mi...

That Bitly link is one I just created that redirects to the Microsoft page.

0 comments

honeybadger13y ago

simonwOP3y ago

I have a paid account too. I'm very confident that it doesn't fetch URLs, despite doing an extremely convincing impression of being able to do so.

Try changing a few characters in that Microsoft URL and see if you still get the same results.

simonwOP3y ago

Wrote more about that here: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...

j / k navigate · click thread line to collapse