Show HN: Selfhostllm.org – Plan GPU capacity for self-hosting LLMs (opens in new tab)

(selfhostllm.org)

7 pointserans7mo ago3 comments

A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.

3 comments

eransOP7mo ago

I also added a Mac version: https://selfhostllm.org/mac/ so you can know which models you can run on your Mac and get an estimated tokens/sec.

I see it doesn't take GPU performance into consideration when showing the estimates. H100 and A100 are performing the same. Am I doing it wrong?

Very useful, thanks. I'm missing a reset button though.

j / k navigate · click thread line to collapse

(selfhostllm.org)

7 pointserans7mo ago3 comments

A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.

eransOP7mo ago

I also added a Mac version: https://selfhostllm.org/mac/ so you can know which models you can run on your Mac and get an estimated tokens/sec.

I see it doesn't take GPU performance into consideration when showing the estimates. H100 and A100 are performing the same. Am I doing it wrong?

Very useful, thanks. I'm missing a reset button though.

j / k navigate · click thread line to collapse