undefined | Better HN

0 pointseurekin4d ago0 comments

If I started today, with building a server, I'd jump right into verified set-ups and writeups, like this one:

You can find info about running a patched version of vllm for 1x24gb, 2x and 4x. There's also quite a few "blackwell" subreddits, where people seem to share a lot of substantial information, if you're going the 6000 route.

0 comments

1 comments · 1 top-level

hypfer4d ago

That writeup is completely unhinged and utterly incomprehensible to follow.

It just throws "you can do <large number>" at you, with no real explainer regarding how it manages that and which trade-offs are made. I still don't know for certain, but I think one of those trade-offs is 3 bit context? Which is a terrible idea.

Please don't share these walls of noise. They shouldn't exist

j / k navigate · click thread line to collapse