undefined | Better HN

0 pointshalJordan8mo ago0 comments

Lpddr5x (not lpddr5) is 10.7 Gbps. Gddr7 is 32 Gbps. So it's going to be slower

0 comments

2 comments · 1 top-level

codedokode8mo ago· 1 in thread

Yes but in matrix multiplication there are O(N²) numbers and O(N³) multiplications, so it might be possible that you are bounded by compute speed.

both are equally important. compute for prefill and mem bandwidth for generation

j / k navigate · click thread line to collapse