undefined | Better HN

0 pointskarxxm2y ago0 comments

You wrote „out of the box“, did you find a way to improve this?

0 comments

3 comments · 1 top-level

You can do PCA or some other dimensionality reduction technique. That’ll reduce computation and improve signal/noise ratio when comparing vectors.

karxxmOP2y ago

Unfortunately this is not feasible with a large amount of words due to the quadratic scaling. But thanks for the response!

minimaxir2y ago

Not sure what you mean by large amount of words. You can fit a PCA on millions of vectors relatively performantly, then inference from it is just a matmul.

1 more reply

j / k navigate · click thread line to collapse