undefined | Better HN

0 pointsjmiskovic2y ago0 comments

It's been confirmed by multiple (unofficial) sources that GPT-4 is 8 models, each 220B parameters. Another rumor is GPT-4 being 16x111B models.

There's a quite fresh and active project replicating something similar with herd of llamas: https://github.com/jondurbin/airoboros

0 comments

Can you provide some of these sources?

All of these should have close ties with OpenAI, but no one from inside confirmed it as far as I know.

j / k navigate · click thread line to collapse

0 pointsjmiskovic2y ago0 comments

It's been confirmed by multiple (unofficial) sources that GPT-4 is 8 models, each 220B parameters. Another rumor is GPT-4 being 16x111B models.

There's a quite fresh and active project replicating something similar with herd of llamas: https://github.com/jondurbin/airoboros

Can you provide some of these sources?

All of these should have close ties with OpenAI, but no one from inside confirmed it as far as I know.

j / k navigate · click thread line to collapse