Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
0 points
Hugsun
2mo ago
0 comments
Save
Share
They're comparing Qwen's moe vs dense (smaller difference) against Gemma's moe vs dense (bigger difference). Your proposed alternative misses the point.
0 comments
1 comments · 1 top-level
top
newest
oldest
zozbot234
2mo ago
Gemma's dense is bigger than its moe's total parameters. You could totally expect the moe to do terribly by comparison.
j
/
k
navigate · click thread line to collapse