undefined | Better HN

0 pointselorant3y ago0 comments

There's an online demo of Vicuna-13b where you can test its efficiency:

0 comments

3 comments · 2 top-level

csdvrx3y ago· 1 in thread

Yes, but can you replicate that functionality using llama.cpp?

If so, what did you run with main?

I haven't been able to get an answer, while for the question above, I can get 'I chose the word "laptop"' with mlc-llm

elorantOP3y ago

For the tasks I need it the efficiency is similar to the online model. Only slower. I don't care for conversational functionality.

After two prompts I was astounded by the innacuracies present in the answers. An they were pretty easy questions.

j / k navigate · click thread line to collapse