Ask HN: Is anybody using llama.cpp for production?

11 pointsHardikVala8mo ago1 comments

Considering the server version of llama.cpp for a commercial use case over bulkier options like vllm. But wondering if it's been battle-tested in production environments.

1 comments

thanhdongnguyen8mo ago

No, i think llama.cpp only using testing model

j / k navigate · click thread line to collapse

Ask HN: Is anybody using llama.cpp for production?

11 pointsHardikVala8mo ago1 comments

Considering the server version of llama.cpp for a commercial use case over bulkier options like vllm. But wondering if it's been battle-tested in production environments.

1 comments

thanhdongnguyen8mo ago

No, i think llama.cpp only using testing model

j / k navigate · click thread line to collapse