Milvus Lite: The Lightweight Version of Milvus (opens in new tab)

(milvus.io)

56 pointsstephen372y ago17 comments

17 comments

15 comments · 4 top-level

ko_pivot2y ago· 8 in thread

Maybe I don’t have enough ‘AI’ experience to understand, but I’m not getting the future of vector databases. 90% of the use cases I’ve encountered also benefit from keyword search, faceting, etc. and therefore a more traditional search engine like Elastic, Meilisearch, or even Postgres makes more sense than something that is purely focused on the vector index. At this point every search engine has a solid vector and hybrid search implementation.

manishsharan2y ago

I have been playing with Milvus but as my use case evolves, I think PGVector may be a better fit . I currently store a lot of enriched data in PG and embeddings in Milvus. Consolidating them into one DB makes sense to me.

mind-blight2y ago

We're using PG vector alongside our other dat. It's has pros and cons. I've found checking to be really slow, so we don't index vectors. We just make sure the query filters down on a small enough subset where a direct comparison is good enough.

The other thing we've encountered is that vectors take up a lot of storage space compared to the normal columns (easily a couple kb per row). You can fill up a db really quickly, especially if you're embedding really small chunks

paul-tharun2y ago

You should take a look at qdrant then. Might fit your use case

spacecadet2y ago

I dont think any (maybe Milvus cool-aid drinkers) would disagree. I recently used Milvus(for the first time) it made sense because it was quick to implement, purpose built, and is working exactly as I intended. Doesn't mean Ill go around blindly using Milvus everywhere. I also like Neo, Duck, Postgres, Parquet, etc etc. Just tools.

menacingly2y ago

definitely overhyped, but not useless. Consider one of your examples, Elastic. It's often employed in situations where the db could handle what it's doing just fine, but it survives, Largely because of optimizations it is free to make knowing it is targeted at a narrow set of tasks.

mewpmewp22y ago

Scaling, accuracy and search quickness is very important for vector dbs. Do other general purpose databases scale as well as specialized ones?

Because ideally they hold massive, well optimised indexes in their memory to be able to search quickly and not miss any vectors.

sa-code2y ago

Vector databases are just the easiest way to make a search engine in a demo.

ukuina2y ago

This is the right answer. When it's time to make a product, you either use a managed VectorDB or ditch it for a more traditional datastore with careful indexing.

1 more reply

syntaxfree2y ago· 3 in thread

So like chromadb

gkapur2y ago

Not really. This is more like SQLite or DuckDB for vector databases (on disk.) Chroma is more like redis for vector databases (in memory.)

We have seen similar products in the olap space, as well, ie. Clickhouse local.

valstu2y ago

Chroma used DuckDB at some point, might not be the case anymore though

mhuffman2y ago

Doesn't DuckDB already do vector search?

1 more reply

nutanc2y ago

This is awesome. Does Milvus lite also support binary embeddings?

mikl2y ago

It’ll be nice when the AI hype settles down a bit, so many of these “re-invent the wheel, with more AI sprinkles” projects popping up.

So many existing DBs can already do vector search, do we really need one dedicated to just that?

1 more reply

j / k navigate · click thread line to collapse

17 comments

15 comments · 4 top-level

ko_pivot2y ago· 8 in thread

manishsharan2y ago

mind-blight2y ago

paul-tharun2y ago

You should take a look at qdrant then. Might fit your use case

spacecadet2y ago

menacingly2y ago

mewpmewp22y ago

Scaling, accuracy and search quickness is very important for vector dbs. Do other general purpose databases scale as well as specialized ones?

Because ideally they hold massive, well optimised indexes in their memory to be able to search quickly and not miss any vectors.

sa-code2y ago

Vector databases are just the easiest way to make a search engine in a demo.

ukuina2y ago

This is the right answer. When it's time to make a product, you either use a managed VectorDB or ditch it for a more traditional datastore with careful indexing.

1 more reply

syntaxfree2y ago· 3 in thread

So like chromadb

gkapur2y ago

Not really. This is more like SQLite or DuckDB for vector databases (on disk.) Chroma is more like redis for vector databases (in memory.)

We have seen similar products in the olap space, as well, ie. Clickhouse local.

valstu2y ago

Chroma used DuckDB at some point, might not be the case anymore though

mhuffman2y ago

Doesn't DuckDB already do vector search?

1 more reply

nutanc2y ago

This is awesome. Does Milvus lite also support binary embeddings?

mikl2y ago

It’ll be nice when the AI hype settles down a bit, so many of these “re-invent the wheel, with more AI sprinkles” projects popping up.

So many existing DBs can already do vector search, do we really need one dedicated to just that?

1 more reply

j / k navigate · click thread line to collapse