undefined | Better HN

0 pointsac291y ago0 comments

Some models are more sensitive to quantization than others, presumably AI Studio is running the full 16 bit model.

Try maybe the 8bit quant if you have the hardware for it? ollama run hf.co/unsloth/gemma-3-27b-it-GGUF:Q8_0

0 comments

I tested the full fp16 gguf

j / k navigate · click thread line to collapse

0 pointsac291y ago0 comments

Some models are more sensitive to quantization than others, presumably AI Studio is running the full 16 bit model.

Try maybe the 8bit quant if you have the hardware for it? ollama run hf.co/unsloth/gemma-3-27b-it-GGUF:Q8_0

I tested the full fp16 gguf

j / k navigate · click thread line to collapse