undefined | Better HN

0 pointstrinix9121y ago0 comments

> 2) the translation models often vary widely between languages so doing well on French doesn't mean they will do well on something else.

Especially this. I often come across AI products that claim to do well in 100+ languages, show some really good DE/FR/SP/RU examples, then I try it with my language (Slovene) and am just disappointed. If you claim to support all those languages, please have a sample result in all of them. Even if they aren't all equally good, it comes across as more genuine than making bold claims that anyone who speaks a language with < 10 million speakers knows likely aren't true.

0 comments

1 comments · 1 top-level

PaulRobinson1y ago

Word Error Rates (WER), are the standardised metric for understanding that. The number is the number of words you would need to add, remove or replace per 100 words (so lower is better). Spanish these days is often down in the 2-3 region, English in the 4 region, and Finnish in the 7-8 region. Slovenian is typically 15-20, and Punjabi, Bengali and other SE Asian languages hit at 40+.

By far the biggest factor seems to be the amount of translated material in that language, followed by how "obvious" the rules of the language are making it easy to decompose algorithmically.

j / k navigate · click thread line to collapse