Show HN: Evaluating Local LLMs as language translators for my app (opens in new tab)

(lector.dev)

4 points3stacks3d ago2 comments

This is my first attempt at running an eval of this nature so would love some methodology feedback.

I can't guarantee the sources weren't already in the model's inputs without getting novel translations from native speakers, but from my experience using the top models, they feel very accurate. Even encountering somewhat obscure texts from a relatively small language the translations generally beat Google Translate for proper idiomatic meaning.

2 comments

2 comments · 1 top-level

robby11103d ago· 1 in thread

Interesting study! First time I've really looked at this sort of study so probably not the best to comment on methodology but the overall premise of using LLMs for language conversion to increase an apps TAM is appealing

3stacksOP3d ago

Thank you! Running phrase translations through an LLM really helps with processing idiomatic phrasing, and also enables quite good output practice. Spoken output practice would be a game changer.

There's definitely a lot of potential in this space especially in generating learner content in languages where there's sparse content online.

j / k navigate · click thread line to collapse