Ah, I see, I understand but I haven't encountered that terminology used that way. I tend to think of one-hot encoding as not a vector space, however, as the information is actually only lg2(10000) per token. However the embedding vector of 512 or however many floats includes lots of information from positionally related tokens, so it's quite a bit more than lg2(vocab).