undefined | Better HN

0 pointsrunako7mo ago0 comments

Relevant to this is that Perplexity lies to the user when specifically asked about this. When the user asks if there is a robots.txt file for the domain, it lies and says there is not.

If an LLM will not (cannot?) tell the truth about basic things, why do people assume it is a good summarizer of more complex facts?

0 comments

charcircuit7mo ago

The article did not test if the issue was specific to robots.txt or if it can not find other files.

There is a difference between doing a poor summarization of data, and failing to even be able to get the data to summarize in the first place.

runakoOP7mo ago

> specific to robots.txt > poor summarization of data

I'm not really addressing the issue raised in the article. I am noting that the LLM, when asked, is either lying to the user or making a statement that it does not know to be true (that there is no robots.txt). This is way beyond poor summarization.

charcircuit7mo ago

I would say it's orthogonal to it. LLMs being unable to judge their capabilities is a separate issue to summarization quality.

1 more reply

j / k navigate · click thread line to collapse

0 pointsrunako7mo ago0 comments

Relevant to this is that Perplexity lies to the user when specifically asked about this. When the user asks if there is a robots.txt file for the domain, it lies and says there is not.

If an LLM will not (cannot?) tell the truth about basic things, why do people assume it is a good summarizer of more complex facts?

0 comments

charcircuit7mo ago

The article did not test if the issue was specific to robots.txt or if it can not find other files.

There is a difference between doing a poor summarization of data, and failing to even be able to get the data to summarize in the first place.

runakoOP7mo ago

> specific to robots.txt > poor summarization of data

charcircuit7mo ago

I would say it's orthogonal to it. LLMs being unable to judge their capabilities is a separate issue to summarization quality.

1 more reply

j / k navigate · click thread line to collapse