undefined | Better HN

0 pointsdeaux3mo ago0 comments

This goes way back. Even back in the 1.5 days it was the best multilingual model, when HN still treated it as entirely uncompetitive all-around. Just because, exactly as you're saying, it's not a core concern of people here. The two fields Gemini models have been number one at for years now are A. multilinguality B. image understanding. At no point since the release of Gemini 1.5 Pro way back has any Anthropic or OpenAI model done performed better at either.

Even those who have zero experience with different (human) languages could've known this if they liked, from the fact that on the LMArena leaderboards, Gemini models have consistently ranked much higher in non-English languages than in English. This gap has actually shrunk a lot over time! In the 1.5 Pro days this advantage was huge, it would be like 10th in English and 2nd in many other languages.

Nevertheless, it still depends on the specific language you're targeting. Gemini isn't the winner on every single one of them. If you're only going to choose one model for use with many languages, it should be Gemini. But if the set of languages isn't too large, optimizing model selection per language is worth it.

0 comments

curioussquirrel3mo ago

In our previous tests, when it was 1.5 Pro against GPT 4o and Claude Sonnet 3.7, Gemini wasn't winning in the multilingual race, but it was definitely competitive. 2.5 and 3.0 seems to be big leaps from the 1.5 days. That said, it also depends on the testing methodology; we tested a bunch of use cases mostly to test core linguistic proficiency. Not as much complex tasks in language or cultural knowledge.

deauxOP3mo ago

Which languages, how popular, how many? The biggest difference has been for low-resource or far-from-English languages. Thai, Korean, Vietnamese, and so on. For something like German or French all of them were of course good enough that general intelligence and other factors overruled any language differences. I didn't take screenshots, maybe archive.org has them, but during the entire period of that generation of models on the LMArena leaderboard there was this large gap between 1.5 Pro rankings on such languages vs on English, which was backed up by our experience including feedback from groups of native speakers.

curioussquirrel3mo ago

And regarding specific models - we obviously only tested a few languages, and there are thousands of them in the world. But Gemini seems to lead the pack basically regardless of the language your throw at it. YMMV.

staticman23mo ago

For translating webnovels My preference is Opus, followed By Gemini, followed by GPT 5.

This is based on informally comparing samples and counting annoyingly big enough to distract the reader mistakes rather than the error rate overall.

GPT 5 seems to be worse at webnovel translation than 4o or 4.1 was, oddly enough.

j / k navigate · click thread line to collapse

0 pointsdeaux3mo ago0 comments

0 comments

curioussquirrel3mo ago

deauxOP3mo ago

curioussquirrel3mo ago

staticman23mo ago

For translating webnovels My preference is Opus, followed By Gemini, followed by GPT 5.

This is based on informally comparing samples and counting annoyingly big enough to distract the reader mistakes rather than the error rate overall.

GPT 5 seems to be worse at webnovel translation than 4o or 4.1 was, oddly enough.

j / k navigate · click thread line to collapse