undefined | Better HN

0 pointssuslik2mo ago0 comments

What is reasonable hardware in your case? Doesn’t this model require 50+ Gb vram?

0 comments

1 comments · 1 top-level

Gemma-4-26B-A4B does not require 50+ Gb of vram. It is a MoE model so only 4B of active parameters at a time and not as GPU dependent. I can run it on 16gb of vram and ~20gb of DDR5 regular ram for a 8 bit quant.

j / k navigate · click thread line to collapse