undefined | Better HN

0 pointsbaobabKoodaa2y ago0 comments

This is a common approach, for example, in data science competitions. Why? Well, if you want to maximize the model's abilities, this is what you have to do. (Not saying Llama 2 is released like this; it probably isn't)

0 comments

3 comments · 1 top-level

snowstormsun2y ago· 2 in thread

Yeah but in competitions there's a secret test set used to evaluate the model.

baobabKoodaaOP2y ago

I have personally shipped "untested" models in production in situations where a "secret test set" does not exist. (Train on subset of data -> evaluate on different subset of data -> train again on entire dataset).

I do not consider myself to be insane.

snowstormsun2y ago

I didn't mean to insult anyone. The idea of not knowing the actual performance of the model just intuitively seems to me like it's a bit of a gamble. I have only trained models in a scientific context before, where this was never an option.

1 more reply

j / k navigate · click thread line to collapse