undefined | Better HN

0 pointspants21y ago0 comments

Not really, MoE is trained all at once and the 'experts' don't have pre-defined specializations. They end up being more like "punctuation expert" and "pronoun expert" than "math expert" and "french expert"

0 comments

laborcontract1y ago

Haven't tried any yet, but it sounds like parent may be interested in an LLM router. https://github.com/lm-sys/RouteLLM

j / k navigate · click thread line to collapse

0 pointspants21y ago0 comments

Not really, MoE is trained all at once and the 'experts' don't have pre-defined specializations. They end up being more like "punctuation expert" and "pronoun expert" than "math expert" and "french expert"

0 comments

laborcontract1y ago

Haven't tried any yet, but it sounds like parent may be interested in an LLM router. https://github.com/lm-sys/RouteLLM

j / k navigate · click thread line to collapse