Show HN: Chroma Cloud – serverless search database for AI (opens in new tab)

Many ways potentially - but one way is Chroma makes all this pain go away.

We're also working on some ingestion tooling that will make it so you don't have to scale, manage or run those pipelines.

Chroma is fully OSS - embedded, single-node and distributed (data and control plane). afaik lance distributed is not OSS.

We do have plans to release the crate (enabling embedded chroma in rust) - but haven't gotten around to it yet. Hopefully soon!

> Do you see all providers converging on similar alpha i.e cheap object storage, nvme drives,ssd cache to solve this?

It's not only a new pattern in search workloads, but it's happening in streaming, KV, OLTP, OLAP, etc. Yea - it's the future.

I think there's a misconception among many non-technical people that they should "fine tune" models - to make an LLM sound like their brand or know their internal data. But, the best practice today is "context engineering" - to use a commodity LLM, and add propriety information into the prompt.

The hard part of context engineering is knowing which data to incorporate. For example, all help docs in every context creates context rot [1] that hurts accuracy, so you need a way to find which information to put in which context. The solution is essentially an internal search engine, which is Chroma.

Chroma gives AI multiple ways to search, and you can either hard-code which one works best for you - or let an agent write its own queries to conduct research by itself. Vector search is a generation ahead of old search infrastructure, and is useful for relatedness queries - like "help docs about billing". Full-text search is useful for proper nouns, like "Next.js". Regex search is useful for code and laws. Metadata search is more nuanced, but becomes really important in document search (e.g., PDFs). Chroma lets you run all of these search methods against private data, and you can use it to even include citations in results.

So, the high-level answer is: Chroma enables you to incorporate your business or customer data into AI.

[1] https://research.trychroma.com/context-rot

"Sell the sizzle, not the steak" is a real thing for a reason.

I understand your point. Chroma Cloud has been quietly live in production for a year, and we have been discussing this architecture publicly for almost two years now. You can see this talk I gave at the CMU databases group - https://youtu.be/E4ot5d79jdA?si=i64ouoyFMevEgm3U. Some details have changed since then. But the core ideas remain the same.

The business model similarities mostly fall out of our architecture being similar, which mostly falls out of our constraints with respect to the workload being the same. There are only so many ways you can deliver a usage based billing model that is fair, understandable, and predictable. We aimed for a billing model that was all three, and this is what we arrived at.

On aesthetics, that’s always been our aesthetic, I think a lot of developer tools are leaning into the nostalgia of the early PC boom during this AI boom (fun fact, all the icons on our homepage are done by hand!).

On differences, we support optimized regexes vs full-scans, lending better performance. We also support trigram based full-text search which can often be useful for scenarios which need substring matches. We also support forking, which allows for cheap copy-on-write clones of your data, great for dataset versioning and tracking git repos with minimal cost. We've been building with support for generic sparse vectors (in beta) which enables techniques like SPLADE to be used, rather than just BM25. You can also run Chroma locally, enabling low-latency local workflows. This is great for AI apps where you need to iterate on a dataset until it passes evals, and then push it up to the cloud.

Chroma is Apache 2.0 open source - https://github.com/chroma-core/chroma and has a massive developer community behind it. Customers can run embedded, single-node and distributed Chroma themselves. We've suffered from depending on closed-source database startups and wanted to give developers using Chroma confidence in the longevity of their choice.

Lastly, we are building with AI workloads front and center and this changes what you build, how you build it and who you build for in the long term. We think search is changing and that the primary consumer of the search API for AI applications is shifting from human engineers, to language models. We are building some exciting things in this direction, more on that soon.

cloud has been in private beta for a year now.

we chose to not release it to the public until we were extremely confident in the system and its characteristics.

databases are a serious business. developers trust us with their mission critical data.

Chroma and Weaviate are both popular OSS projects, 21k and 14k stars respectively.

Weaviate has some features that Chroma does not (yet) - notably vector quantization. Weaviate is a single-node DB that can be run in cluster mode.

Chroma focuses on reliability and operational simplicity. The fully distributed architecture means there are no knobs to tune or config to set.

There's no right tool for everyone. Try both!

Chroma supports multiple search methods - including vector, full-text, and regex search.

Four quick ways Chroma is different than pgvector: Better indexes, sharding, scaling, and object storage.

Chroma uses SPANN (Scalable Approximate Nearest Neighbor) and SPFresh (a freshness-aware ANN index). These are specialized algorithms not present in pgvector. [1].

The core issue with scaling vector database indexes is that they don't handle `WHERE` clauses efficiently like SQL. In SQL you can ask "select * from posts where organization_id=7" and the b-tree gives good performance. But, with vector databases - as the index size grows, not only does it get slower - it gets less accurate. Combining filtering with large indexes results in poor performance and accuracy.

The solution is to have many small indexes, which Chroma calls "Collections". So, instead of having all user data in one table - you shard across collections, which improves performance and accuracy.

The third issue with using SQL for vectors is that the vectors quickly become a scaling constraint for the database. Writes become slow due to consistency, disk becomes a majority vector indexes, and CPU becomes clogged by re-computing indexes constantly. I've been there and ultimately it hurts overall application performance for end-users. The solution for Chroma Cloud is a distributed system - which allows strong consistency, high-throughput of writes, and low-latency reads.

Finally, Chroma is built on object storage - vectors are stored on AWS S3. This allows cold + warm storage tiers, so that you can have minimal storage costs for cold data. This "scale to zero" property is especially important for multi-tenant applications that need to retain data for inactive users.

[1] https://www.youtube.com/watch?v=1QdwYWd3S1g

dedicated solutions have more advanced search features enable more accurate results. search indexing is resource intensive and can contend for resources with postgres/redis. the cost and speed benefits are naturally more pronounced as data volume scales.

for example - chroma has built in regex+trigram search and copy-on-write forking of indexes. this feature combo is killer for the code-search use case.

I see your email in your profile, so I'll reach out.

We work closely with any startup that wants help, ranging from whiteboarding architectures to a shared slack channel.

Single-node Chroma is very easy to self-host for free: https://docs.trychroma.com/guides/deploy/docker

It's a great model for people who want to have grasp beyond the typical software reach.

https://en.m.wikipedia.org/wiki/Gratis_versus_libre

The main reason we have seen people switch from Qdrant to Chroma is for operational simplicity and reliability. With Qdrant you have to size nodes, manage sharding, be oncall, etc - with Chroma there is literally 0 config or operational pain. Search, that just works!