undefined | Better HN

0 pointsigor471y ago0 comments

Open AI fails here:

https://chatgpt.com/share/6770c547-2f90-8004-ba41-21bfa4d3a7...

Curious about your local LLM usage -- do you have that documented, or can you recommend sources on how to get started in that domain? I self host most of my infrastructure, but not LLMs so far. Do you need special hardware? How do you interact with the LLMs? How to you keep them updated? Do you fine tune/do any training, or just of the shelf llama? Do you need to know a bunch about quantization? How fast are the responses? Can you use them in your IDE as a coding assistant? How is resource utilization?

0 comments

2 comments · 2 top-level

Abishek_Muthian1y ago

Update:

I have documented how I run local LLMs here - https://abishekmuthian.com/how-i-run-llms-locally/ .

Abishek_Muthian1y ago

I will document it and share with you.

j / k navigate · click thread line to collapse