undefined | Better HN

0 pointsanon3738391y ago0 comments

I’d recommend trying it on Google AI Studio (aistudio.google.com). I am getting exceptional results on a handful of novel problems that require deep domain knowledge and structured reasoning. I’m not able to replicate this performance with Ollama, so I suspect something is a bit off.

0 comments

tarruda1y ago

Same experience here: On AI Studio, this is easily one of the strongest models I have used, including when compared to proprietary LLMs.

But ollama and openwebui performance is very bad, even when running the FP16 version. I also tried to mirror some of AI studio settings (temp 1 and top p 0.95) but couldn't get it to produce anything useful.

I suspect there's some bug in the ollama releases (possibly wrong conversation delimiters?). If this is fixed, I will definitely start using Gemma 3 27b as my main model.

anon373839OP1y ago

Update: Unsloth is recommending a temperature of 0.1, not 1.0, if using Ollama. I don’t know why Ollama would require a 10x lower value, but it definitely helped. I also read some speculation that there might be an issue with the tokenizer.

pzo1y ago

Maybe model is sensitive to quantization, by default ollama quantize it significantly.

tarruda1y ago

I tried ollama fp16 and it had the same issues.

j / k navigate · click thread line to collapse

0 comments

tarruda1y ago

Same experience here: On AI Studio, this is easily one of the strongest models I have used, including when compared to proprietary LLMs.

I suspect there's some bug in the ollama releases (possibly wrong conversation delimiters?). If this is fixed, I will definitely start using Gemma 3 27b as my main model.

anon373839OP1y ago

pzo1y ago

Maybe model is sensitive to quantization, by default ollama quantize it significantly.

tarruda1y ago

I tried ollama fp16 and it had the same issues.

j / k navigate · click thread line to collapse