undefined | Better HN

0 pointsRhapso2y ago0 comments

FTR LLM embeddeds are a lot less efficient than ASCII for communication and are likely to stay that way.

0 comments

4 comments · 1 top-level

riku_iki2y ago· 3 in thread

absolutely not. Transformer layers already communicate using embeddings, and ASCII would be absolutely less efficient there.

RhapsoOP2y ago

And how many bits are in an embedded vector?

12k for gpt3.

It is not bits, but weights

j / k navigate · click thread line to collapse