Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
0 points
ein0p
2y ago
0 comments
Save
Share
Nope. Just the weights would take 88GB at 4 bit. 128GB MBP ought to be able to run it. If I were to guess, a version for Apple MLX should be available within a few days, for those of us fortunate enough to own such a thing.
0 comments
1 comments · 1 top-level
top
newest
oldest
Art9681
2y ago
It’s already available. I had it running yesterday morning in an M3 MAX 128GB. I get about 6tps.
https://www.reddit.com/r/LocalLLaMA/s/MSsrqWHYga
j
/
k
navigate · click thread line to collapse