undefined | Better HN

0 pointspogue1y ago0 comments

The APK doesn't include any of the models, so I'm not sure where you get them, where you're supposed to put them on the phone (same directory? Do they need to be compiled?)

When I ask it a question it doesn't respond & when you attempt to switch models it starts looking for nearby devices.

It also appears to be role play for some characters, so it's really not a GenAI chat bot, as far as I can tell. https://ibb.co/DRZhZcH

0 comments

sottol1y ago

You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as I said, not working on Pixel 4. Should get a new screen tomorrow and can't wait to try on my Pixel 6 Pro.

Eg "phi-3 gguf" eventually take you here [1]. and you can download the Q4 quantized model on that page.

[1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...

pogueOP1y ago

I see, ty. I'm not familiar with quantization. Can you give me a brief explainer for that or point me in the right direction?

j / k navigate · click thread line to collapse