Demo: https://youtu.be/2g4BxTZ9ztg
Two years ago, Yuhong and I had the same recurring problem. We were on growing teams and it was ridiculously difficult to find the right information across our docs, Slack, meeting notes, etc. Existing solutions required sending out our company's data, lacked customization, and frankly didn't work well. So, we started Danswer, an open-source enterprise search project built to be self-hosted and easily customized.
As the project grew, we started seeing an interesting trend—even though we were explicitly a search app, people wanted to use Danswer just to chat with LLMs. We’d hear, “the connectors, indexing, and search are great, but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them”.
Many users would add RAG, agents, and custom tools later, but much of the usage stayed ‘basic chat’. We thought: “why would people co-opt an enterprise search when other AI chat solutions exist?”
As we continued talking to users, we realized two key points:
(1) just giving a company secure access to an LLM with a great UI and simple tools is a huge part of the value add of AI
(2) providing this well is much harder than you might think and the bar is incredibly high
Consumer products like ChatGPT and Claude already provide a great experience—and chat with AI for work is something (ideally) everyone at the company uses 10+ times per day. People expect the same snappy, simple, and intuitive UX with a full feature set. Getting hundreds of small details right to take the experience from “this works” to “this feels magical” is not easy, and nothing else in the space has managed to do it.
So ~3 months ago we pivoted to Onyx, the open-source chat UI with:
- (truly) world class chat UX. Usable both by a fresh college grad who grew up with AI and an industry veteran who’s using AI tools for the first time.
- Support for all the common add-ons: RAG, connectors, web search, custom tools, MCP, assistants, deep research.
- RBAC, SSO, permission syncing, easy on-prem hosting to make it work for larger enterprises.
Through building features like deep research and code interpreter that work across model providers, we've learned a ton of non-obvious things about engineering LLMs that have been key to making Onyx work. I'd like to share two that were particularly interesting (happy to discuss more in the comments).
First, context management is one of the most difficult and important things to get right. We’ve found that LLMs really struggle to remember both system prompts and previous user messages in long conversations. Even simple instructions like “ignore sources of type X” in the system prompt are very often ignored. This is exacerbated by multiple tool calls, which can often feed in huge amounts of context. We solved this problem with a “Reminder” prompt—a short 1-3 sentence blurb injected at the end of the user message that describes the non-negotiables that the LLM must abide by. Empirically, LLMs attend most to the very end of the context window, so this placement gives the highest likelihood of adherence.
Second, we’ve needed to build an understanding of the “natural tendencies” of certain models when using tools, and build around them. For example, the GPT family of models are fine-tuned to use a python code interpreter that operates in a Jupyter notebook. Even if told explicitly, it refuses to add `print()` around the last line, since, in Jupyter, this last line is automatically written to stdout. Other models don’t have this strong preference, so we’ve had to design our model-agnostic code interpreter to also automatically `print()` the last bare line.
So far, we’ve had a Fortune 100 team fork Onyx and provide 10k+ employees access to every model within a single interface, and create thousands of use-case specific Assistants for every department, each using the best model for the job. We’ve seen teams operating in sensitive industries completely airgap Onyx w/ locally hosted LLMs to provide a copilot that wouldn’t have been possible otherwise.
If you’d like to try Onyx out, follow https://docs.onyx.app/deployment/getting_started/quickstart to get set up locally w/ Docker in <15 minutes. For our Cloud: https://www.onyx.app/. If there’s anything you'd like to see to make it a no-brainer to replace your ChatGPT Enterprise/Claude Enterprise subscription, we’d love to hear it!
Can you call it open source if you need a subscription license to run / edit the code?
Content under backend/ee requires a license, everything else is MIT Expat. Pretty standard stuff.
> Can you call it open source if you need a subscription license to run / edit the code?
MIT is open source, their other stuff isn't. Pretty clear.
Could you elaborate why this approach is confusing?
A common "trick" for commercial open source software is to use a copyleft license, which restricts redistribution as part of commercial products, and to offer a paid license to get around that.
As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness. It is open code for sure but it is a business and so incentive is to run it like a business which will conflate with how the project is used by the community.
Btw, there is nothing wrong with that but let's be honest here if you get this funded (perhaps it already is) who are you going to align your mission with - the open source community or shareholders? I don't think you can do both. Especially if a strong competitor comes along that simply deploys the same version of the product. We have seen this story many times before.
Now, this is completely different from let's say Onyx being an enterprise search product where you create a community-driven version. You might say that fundamentally it is the same code but the way it is presented is different. Nobody will think this is open-source but more of "the source is available" if you want to check.
I thought perhaps it will benefit to share this prospective here if it helps at all.
Btw, I hear good things about Onyx and I have heard that some enterprises are already using it - the open-source version.
It's an MIT license. That IS open source.
If they have a commercial strategy - that's a GoodThing. It means they have a viable strategy for staying in business, and keeping the project maintained.
MIT == OpenSource. Pricing == Sustainable. That's a horse worth backing IMO.
The source is available and you can do much with it, but the incentive is that this alone should not be enough.
At the top level looks like open source but it is not really because parts (the most useful ones) of the project are not. Imagine if python was open source but the core libraries where not. You wont call this open source in the true spirit of open source. You could make the argument that at least it is sustainable because they a have now a business model. It doesn't add up.
I prefer more of a honest take on software. There is nothing wrong to make money while contributing back to the community in some meaningful way or by simply being transparent. In fact this is the best kind and there are plenty of good examples.
All I am saying is that when I see such projects I tend to think that in most cases they are dishonest to themselves or their communities or both.
I think there can be other valid perspectives than your own.
For most people, the chat is the entrypoint to LLMs, and people are growing to expect more and more. So now it might be basic chat, web search, internal RAG, deep research, etc. Very soon, it will be more complex flows kicked off via this interface (e.g. cleaning up a Linear project). The same "chat UI" that is used for basic chat must (imo) support these flows to stay competitive.
On the engineering side, things like Deep Research are quite complex/open-ended, and there can be huge differences in quality between implementations (e.g. ChatGPTs vs Claude). Code interpreter as well (to do it securely) is quite a tricky task.
That being said, I think there is an opportunity for them to discover and serve an important enterprise use case as AI in enterprise hits exponential growth.
Something like this could have a nice future as an open source chat framework for building custom UIs if it's well made and modular, but that isn't gonna work well with a SaaS model.
"What's Max's GitHub username?" "I need wire transfer instructions for an incoming wire"
We also index competitors' helpdesks and KB articles to track new features they're rolling out. Our tech support team uses it daily because Freshdesk's AI is terrible and their internal KB search is lackluster. Onyx actually finds things. The value isn't in being "one chat to rule them all" — it's in unified search across disparate systems with citations. That's not getting commoditized anytime soon. Keep up the good work, team.
Ooooof. Careful with that.
It does requires having UI components for many different types of interactions (e.g. many ways to collect user input mid-session + display different tools responses like graphs and interactives). With this, people should be able to easily build complex tools/flows on top of that UI, and get a nice, single interface (no siloed tools/swapping) for free. And having this UI be open-source make this easier.
I would actually argue chat windows are terrible ui/ux for most cases and users. It does the opposite of `don't make me think`. Too much potential for user error.
Not saying there shouldn't be any LLM integration/features, just that it should be in the form of a button press or something (familiar ux), not the same chatgpt interface that all the early apps are trying to mimic for no good reason.
If you're a non-tech company, why doesn't your org dictate a single model provider? How do these decisions work internally, and how to the departments consume them? (Are they consuming the tools?)
> make their own complicated agents.
Asking a non-tech employee to make an agent sounds like hell.
> we only put up with it because it has access to email, SharePoint and Teams.
Ah, that's how a third party can make money. Bake in external org-wide knowledge and enable search.
Also, open-source works really here, since connectors are a long-tail game. We've tried to make it easy to add connectors (a single python interface), and as a result over half of our connectors are contributed by the community. We expect that percentage to grow over time. This means that compared to something like Agentspace, we'll very likely be connected to all of the key tools at your company (and if we aren't, you can easily add an integration).
I also wouldn't pin it as chat app specific. Quite a few VC funded open core software has adopted that pattern post ~2020(?): cal.com, Dagster, Gitlab
Thanks, lawyers, you make everything better!
Ideally not "open"router? It's not open, and don't they charge a margin?
Personally, I use Raycast for all my personal work
- Instability when self-hosting
- Hard to get in touch with sales when looking for SLA-based contracts
- Cluttered product; Multiple concepts seemingly serving the same purpose (e.g. function calling vs. MCP); Most pre-MCP tools suffer from this
- Trouble integrating it with OIDC
- Bad docs that are mostly LLM generated
Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.
Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.
Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).
Onyx Devs: This looks awesome, I will definitely add it to my list of things to try out... close to the top! Thanks, and please keep it cool!
I was told there would be rapid prototyping with AI. Haven't seen any of the above.
> but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them
I did get a little giggle out of that because I've never heard anyone say that hooking up 3rd party llms to anything was any way secure.
The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly.
1/ large connector suite + good RAG. Answers at scale is hard, and from our enterprise search roots, we've spent a lot of time with it. It's something that many teams expect from their chat UI.
2/ deep research + open-source code interpreter.
3/ simpler UX. LibreChat has a lot of customizability exposed front and center to the user, which is great for the power user but can be overwhelming for someone new to using AI systems.
Things like shortcuts to access the chat + ability to reach into the local file system are very compelling.
It’s nice to see an attempt at an end to end stack (for all that it seems this is “obvious” … there are not that many functional options) but wow we’ve forgotten the basis of making useful products. I’m hoping it gets enough time to bake.
The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left.
In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat.
I'm sure you guys are thinking about this, but please just go through the steps of setting up via docker, uploading say a grad student's worth of papers and docs, scrape a small topic off wikipedia, try and use it for three days and take a look at the ergonomics. It's not easy to regroup sets of documents, get results that link to the document to view post indexing for RAG etc. etc. etc.
In general there are a lot of low hanging RAG optimizations that you could do to make this usable for people who don't want to write their own bits of code to make it usable. I ended up fiddling a bit more with anythingllm which, while having fewer features, understands the workflows a bit more.
I understand they're trying to attract enterprisy customers, but even some of those are probably going to want to try it out first. Would be nice to have an easy minimal install option that doesn't require a deep dive into the project to figure out.
We should definitely update the Local guide to reflect the ^, thanks for pointing that out.
For the container footprint - pretty much all of them are necessary to run locally. We hope that just a large # of containers shouldn't be a problem, as long as they are all lightweight (e.g. total resource utilization is low). We of course could bundle them together into a single "mega container", but that seems against the best practices of Docker.
On a related note, we are looking to have a version that doesn't require the vector database (the heaviest part of the system in terms of resource utilization). Our goal is to have a deployment mode with less than 1gb of RAM required.
For example, Openwebui can be run with just a sqlite database and a backend. Why is nginx needed, or Minio on a single machine with a nice local file system? But I also understand it takes more work to support multiple service configurations so please accept the criticism as constructive (and it's more a general observation of what I've noticed over the past few years).
My main gripe with openwebui, in addition to it being slow is the fact that it mangles documents in the OCR step. tables that could have been understood great by an multi modal llm, just gets mangled by the ocr and lost instead of storing both a text and original representation.
Being able to properly searcbin the knowlege base lime the llm does, but manually would be nice (like get recommendations for docs to add).
My usecase is mostly writing, so having a integrated document refinery editor is also a nice feature list.
I'm probably rambling but these are my base use-cases for a llm ui I personally have found.
Writing is a really common use case, and something we'd like to explore more. Currently people often use Onyx for "write something combining X, Y, and Z documents", but I feel that's just scratching the surface.
But stoked to get alternatives to the area, will try it out once i get time soon.
a mobile application that has parity on the same features that ChatGPT and Claude does...
The main thing I really care about is voice mode, as that's my far preferred way of interacting with LLMs for longer backs and forths (most apps I've seen disable a lot of other functionality during it, which I hate, btw).
Two other things I would like to see are canvas mode and scheduled actions (with decision making capability - e.g. "send a notification if X happens").
I assume such features are going to continue being invented, so I find extensibility to be a huge deal. So much so that one thing I could imagine going really well would be a UI on top of Langchain, which already has most of the facilities for that!
Totally makes sense! We've defined simple `Connector`, `Tool` and (soon) `Agent` interfaces to make it easy to plug in your own implementations/apps. If you wanted, you could just use Langchain under our Agent class to build arbitrary flows.
Additionally, the main chat is created from a series of `Renderer` components, and it should be easy to build your own.
Do you think that's in-line with what you're thinking of, or do you want to build outside of those confines?
> canvas mode and scheduled actions
Yes, writing and recurring are two big areas we want to tackle next year.
> The main thing I really care about is voice mode
Interesting. Why do you think that is?
It definitely sounds good. Of course it's hard to anticipate what interfaces new features will require, it probably won't always be possible to integrate new features with a generic plugin (voice mode is most likely such a feature), but if the code is well architected, this should be okay.
> Interesting. Why do you think that is?
Writing has a lot of friction to me. It's much more comfortable for me to provide context through verbal rambling, which LLMs are great at processing. I like to research stuff and throw around ideas while pacing, doing chores or just lying with my eyes closed.
Unfortunately I just discovered that I won't be able to run Onyx on my low powered home server anyway (https://docs.onyx.app/deployment/getting_started/resourcing#...). I understand that a Vector database requires significant resources to run, but I wish there was a version without it.
Why do we have to yet again poorly copy an oversimplified UI?
The value of local models comes from their huge amount of settings/control that they offer. Why must we throw that all away?
Yet again, the world waits for good UI/UX for pro/prosumers with AI systems. No one is learning from ComfyUI, Automatic1111, or SillyTavern. No, LM-Studio is not actually prosumer
For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice.
The target customers are enterprises where 70% of the target users have never used an AI system before and have to go through a AI training before being allowed access to the system.
Just curious. Especially with both OpenAI and Anthropic really also outpacing startups in release cadence unlike previous cycles.
Guessing your selling point is any model no locking (Assuming we are happy with the privacy SOC 2 etc guarantees on enterprise contracts here)
1/ No model lock-in / ability to use the ideal model for each use case.
2/ More connectivity. A fuller connector library (contributed to by the open-source community). More built-in tools (similar to the ^).
3/ Customizability and flexibility. If you really need a feature, you can build it rather than waiting months (years?) for your request to go through.
4/ White-labeling. You can make it feel like a product built for/by your company rather than generic.
My main concern is how well do all the extra features work compared to the native versions? Like web search, RAG on a document, or deep research, adding images, voice chats - my understanding is the models providers don't provide any API's for any of this stuff, so you have your own implementations of all the extra stuff right? Usually I find the open source versions of all these features aren't up to par with the corporate versions and they lag behind in development.
RAG specifically is our speciality - we've done a ton of optimizations there (hybrid search, document age-based weighting, giving the LLM the ability to read more from interesting docs and less of irrelevant docs, etc.) and we outperform the implementation within ChatGPT quite substantially in internal blind testing.
Curious what you find if you compare them head to head though!
One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise).
Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example).
Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors.
Might have to try this out. OWUI's lagging docs has made managing my own self hosted instance a pain.
PS: Your _See All Connectors_ button on the homepage is 404ing.
"Agents" is a particular area where we feel like we're better than the alternatives (especially if you want something that effectively calls multiple tools in sequence). Curious to hear your thoughts after trying it out!
And this is all made all the more important when supporting the wide range of models, even "weaker" open-source models.
In our opinion, it's a bit silly to build completely in house when you can take something like Onyx as the starting point and be >95% of the way there + have a tons of bells and whistles built in.
Curious, it's a crowded space with other enterprise search companies like Glean and Elastic, and other companies coming in like Notion and Slack.
Why should a prospect choose Onyx over the others?
- "pure chat" experience. From our community (and personal use), we've observed that most queries don't actually involve enterprise search. They much more likely to just require the LLMs internal knowledge (or web search / code execution). Compared to all the companies you've mentioned, we've spent a lot more time refining this more common flow.
- Larger connector suite. As soon as one key source isn't connected, the trustworthiness of the system is dramatically decreased. You second guess "is the info needed to answer this question in there?" for every question. We have a community who builds out connectors for themselves, and then contribute it back for everyone to use. This allows us to cover the long-tail better than companies like Notion and Slack.
- Customizability. An open-source application is the perfect middle ground between a SaaS offering and building blocks. A SaaS option doesn't allow for any customization (we have many customers who have contributed back ux enhancements, small features like guardrails, or enhanced configurations that their users want). Building blocks demand too much domain expertise (search, frontend/UX, ...) for it to be realistic for companies to build something great.
---
Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows. Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.
Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.
Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).
Can you clarify the license and if this actually meets the definition of Open Source as outlined by the OSI [1] or if this is actually just source available similar to OpenWebUI?
Specifically can / does this run without the /onyx/backend/ee and web/src/app/ee directories which are licensed under a proprietary license?
We have https://github.com/onyx-dot-app/onyx-foss, for a fully MIT licensed version of the repo if you want to be safe about the license/feel freedom to modify every file.
However it doesn't seem to have MinerU as a supported backend, which is hands-down the best PDF extraction tool I've ever used (and is self-hostable on a machine with a modest GPU). Could it be added?
Would love to hear your feedback after you get a chance to try Onyx out.
How we think about it: the chat product should be completely open-source and free (forever). To that end we've moved features like SSO (that used to be "enterprise") to be MIT licensed. The chat interface is something pretty much every team needs (be it a proprietary or open-source solution). You can think of this like Apache Spark for Databricks or Ray for Anyscale.
Also, as other folks have pointed out in the thread, there are quite a few other open source options out there. So there's a a ton of outside pressure for our open-source only offering to be very competitive. We hope this reduces the "enshitification" risk that you speak of.
"The reason is because VC needs to show that their flagship investments have "traction" so they manufacture ecosystem interest by funding and encouraging ecosystem product usage. It's a small price to pay. If someone builds a wrapper that gets 100 business users then token use on the foundation layer gets that passed down. Big scheme."
I'm nitpicking, congrats on the launch!
How do you plan to make money? I'm very serious about this question.
I'm in an adjacent (but highly non-LLM field) and I'm grappling with this myself.
Selling compute or being a middle man and paying for API use seems like a low-margin game. It'd have to be with giving convenient search access to org data or something along those lines? Perhaps expanding into some kind of agent product?
My creativity on making money with LLMs is pretty sparse as I'm in the graphics world. But I'm certainly interested in "making money on aggregation" ideas.
Also, are you worried about competitors forking your code?
its interesting to see how "lock-in" is the main pitch here. all things considered, i don't think "lock-in" is relevant at all unless the activity performed with the tool is highly strategic to the company.
you could argue that some orgs may not want openai/anthropic to have their sensitive data leave the parameter, but im also here to tell you that even the most privacy sensitive companies in the world probably resolve this by having a proxy in between the users and the LLM APIs from the labs.
so where does this leave you ? cost savings from OSS? maybe, but its hard to imagine that we are in the phase of the adoption cycle where companies have become as acutely aware of costs as you think they are.
my 2c - focus on the integrations and see which one gets most traction. that will be your value capture mechanism long-term.
- can't fold/unfold code
- lack syntax highlight for some languages (Zig, Odin, D)
We are building a competing open source tool[0] with a very similar focus (strongly relying on interoperable standards like MCP; built for enterprise needs, etc.), though bootstrapping with customers rather than being VC funded. It's nice to see a competitor in the field following similar "OSS Friends" principles, while many of the other ones seem to have strong proprietary tendencies.
(Small heads up: The "view all integrations" button goes to a 404)
And no one bothered to say anything to them?
