undefined | Better HN

0 pointstarruda1mo ago0 comments

I'm only interested in the local, single user use case. Plus I use a Mac studio for inference, so vLLM is not an option for me.

0 comments

mycall1mo ago

You can get concurrency gains [0] as local/single user (multi-agent) use case with vLLM with your Mac Studio.

j / k navigate · click thread line to collapse

0 pointstarruda1mo ago0 comments

I'm only interested in the local, single user use case. Plus I use a Mac studio for inference, so vLLM is not an option for me.

mycall1mo ago

You can get concurrency gains [0] as local/single user (multi-agent) use case with vLLM with your Mac Studio.

j / k navigate · click thread line to collapse