Understanding Context Windows (opens in new tab)

(rkayg.com)

2 pointsrkayg7mo ago2 comments

2 comments

I think degraded performance as the context window increases is one of the largest issues models are facing today. Do you think we've reached the end of any "RAM" based context performance and that external storage/ retrieval mechanisms is the only way forward?

rkaygOP7mo ago

I think we're in the early innings of RAM implementations. I also think RAM and external storage / retrieval are kind of the same thing in that efficient RAM implementation works best w/ another storage system. I hope to do a deeper dive here, but in the mean time check out https://arxiv.org/abs/2310.08560

j / k navigate · click thread line to collapse

2 comments

johndonavon7mo ago

rkaygOP7mo ago

j / k navigate · click thread line to collapse