undefined | Better HN

0 pointsHugsun24d ago0 comments

They're comparing Qwen's moe vs dense (smaller difference) against Gemma's moe vs dense (bigger difference). Your proposed alternative misses the point.

0 comments

zozbot23424d ago

Gemma's dense is bigger than its moe's total parameters. You could totally expect the moe to do terribly by comparison.

j / k navigate · click thread line to collapse

0 comments

zozbot23424d ago

Gemma's dense is bigger than its moe's total parameters. You could totally expect the moe to do terribly by comparison.

j / k navigate · click thread line to collapse