undefined | Better HN

0 points_aavaa_10d ago0 comments

For an fp4 quantization it should fit with room to spare for KVCache

0 comments

2 comments · 1 top-level

Tepix10d ago· 1 in thread

Aren't Macbooks limited to 128GB RAM?

FP4 would require >350GB RAM + KV cache, so no.

_aavaa_OP10d ago

Oops, you’re right. My brain understood Mac Studio for some reason.

j / k navigate · click thread line to collapse