undefined | Better HN

0 pointsjjmarr5mo ago0 comments

Seeing a task-specific model be consistently better at anything is extremely surprising given rapid innovation in foundation models.

Have you tried Aristotle on other, non-Lean tasks? Is it better at logical reasoning in general?

0 comments

3 comments · 1 top-level

runeblaze5mo ago· 2 in thread

Is it though? There is a reason gpt has codex variants. RL on a specific task raises the performance on that task

jjmarrOP5mo ago

Post-training doesn't transfer over when a new base model arrives so anyone who adopted a task-specific LLM gets burned when a new generational advance comes out.

runeblaze5mo ago

Resouce-affording, if you are chasing the frontier of some more niche task you redo your training regime on the new-gen LLMs

j / k navigate · click thread line to collapse