undefined | Better HN

0 pointsfulafel1y ago0 comments

No.

0 comments

Is this paper wrong? - https://arxiv.org/abs/2311.09807

It shows that if you deliberately train LLMs against their own output in a loop you get problems. That's not what synthetic data training does.

belter1y ago

I understand and appreciate your clarification. However would it not be the case some synthetic data strategies, if misapplied, can resemble the feedback loop scenario and thus risk model collapse?

j / k navigate · click thread line to collapse

0 comments

belter1y ago

Is this paper wrong? - https://arxiv.org/abs/2311.09807

simonw1y ago

It shows that if you deliberately train LLMs against their own output in a loop you get problems. That's not what synthetic data training does.

belter1y ago

I understand and appreciate your clarification. However would it not be the case some synthetic data strategies, if misapplied, can resemble the feedback loop scenario and thus risk model collapse?

j / k navigate · click thread line to collapse