undefined | Better HN

0 pointswhimsicalism2y ago0 comments

They have not. You probably read "MoE" and some pop article about what that means without having any clue.

0 comments

2 comments · 1 top-level

matsemann2y ago· 1 in thread

If you know better it would be nice of you to provide the correct information, and not just refute things.

gpt-4 is a sparse MoE model with ~1.2T params. this is all public knowledge and immediately precludes the two previous commentators assertions

j / k navigate · click thread line to collapse