undefined | Better HN

0 pointsaerzen1y ago0 comments

LLAMA is an open-weights model. I like this term, let's use that instead of open source.

0 comments

Can a human programmer edit the weights according to some semantics?

It is possible to merge two fine-tunes of models from the same family by... wait for it... averaging or combining their weights[0].

I am still amazed that we can do that.

This is absolutely wild.

Yes. Using fine tuning.

Yes, there is the concept of a "frakenmerge" and folks have also bolted on vision and audio models to LLMs.

j / k navigate · click thread line to collapse

gowld1y ago

Can a human programmer edit the weights according to some semantics?

It is possible to merge two fine-tunes of models from the same family by... wait for it... averaging or combining their weights[0].

I am still amazed that we can do that.

This is absolutely wild.

Yes. Using fine tuning.

Yes, there is the concept of a "frakenmerge" and folks have also bolted on vision and audio models to LLMs.

j / k navigate · click thread line to collapse