undefined | Better HN

0 pointsmaeil1y ago0 comments

> "An AI assistant was tasked with {task}. The relevant information for their task was {context}. Their answer is {answer}. The correct answer should be something like {ground truth}. Is their answer correct?"

If you have a ground truth, what was the purpose of asking the AI assistant for an answer in the first place?

0 comments

4 comments · 3 top-level

senko1y ago· 1 in thread

When you're writing a test, you usually know the correct answer for that specific combination of input parameters.

maeilOP1y ago

Looking back at it, I must have been very tired when I wrote that!

Or maybe I was thinking about cases where the ground truth is difficult to establish.

pletnes1y ago

The cat is dead. The cat is no longer alive. These are equivalent enough, usually, but fails string comparison.

ghm21801y ago

Or like of you did ai voice calling the goal was XYZ did the conversation get, in so many words, to XYZ?

j / k navigate · click thread line to collapse