Local Code Chatbot Running on 2GB RAM (opens in new tab)

(twitter.com)

1 pointsamasad2y ago1 comments

1 comments

There is even some untapped headroom, as they quantized to Q4 instead of using K-quant.

Not to speak of the potential hooked up to a vector db, swapping out LORAs for different languages and such.

j / k navigate · click thread line to collapse

(twitter.com)

1 pointsamasad2y ago1 comments

There is even some untapped headroom, as they quantized to Q4 instead of using K-quant.

Not to speak of the potential hooked up to a vector db, swapping out LORAs for different languages and such.

j / k navigate · click thread line to collapse