undefined | Better HN

0 pointsmrtranscendence2y ago0 comments

> And if these rumors, that GPT-4 is inherently un-deterministic and unreliable, are true then most enterprises are better off finetuning open source LLMs—which are just as capable

Wait, am I misunderstanding you? I feel like I've had a head injury or something, because I've never heard of an open source LLM that's as capable as GPT-4 (in most scenarios).

0 comments

2 comments · 1 top-level

visarga2y ago· 1 in thread

Only on specific domains, these models don't become generalists like GPT-4, they can become task experts for a single task.

famouswaffles2y ago

Fine-tuned MedPalm is worse than GPT-4 on most Medical Challenge Tests. Fine-tuned Minerva is much worse on arithmetic benchmarks.

The LLM space is just different. There's no guarantee a fine-tuned model will beat a bigger generalist one.

j / k navigate · click thread line to collapse