undefined | Better HN

0 pointsbearjaws1mo ago0 comments

I am curious if it reuses the LLM across all tabs, hard to imagine most machines can boot up 1-2 of any 4gb model unless its a more powerful system.

0 comments

2 comments · 1 top-level

nsvd21mo ago· 1 in thread

I think it obviously will, what would be the benefit to spinning up more than one copy?

It should only need to load one copy of the weights, but each tab/site will need a separate context and KV cache.

j / k navigate · click thread line to collapse