Demo: https://youtu.be/2g4BxTZ9ztg
Two years ago, Yuhong and I had the same recurring problem. We were on growing teams and it was ridiculously difficult to find the right information across our docs, Slack, meeting notes, etc. Existing solutions required sending out our company's data, lacked customization, and frankly didn't work well. So, we started Danswer, an open-source enterprise search project built to be self-hosted and easily customized.
As the project grew, we started seeing an interesting trend—even though we were explicitly a search app, people wanted to use Danswer just to chat with LLMs. We’d hear, “the connectors, indexing, and search are great, but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them”.
Many users would add RAG, agents, and custom tools later, but much of the usage stayed ‘basic chat’. We thought: “why would people co-opt an enterprise search when other AI chat solutions exist?”
As we continued talking to users, we realized two key points:
(1) just giving a company secure access to an LLM with a great UI and simple tools is a huge part of the value add of AI
(2) providing this well is much harder than you might think and the bar is incredibly high
Consumer products like ChatGPT and Claude already provide a great experience—and chat with AI for work is something (ideally) everyone at the company uses 10+ times per day. People expect the same snappy, simple, and intuitive UX with a full feature set. Getting hundreds of small details right to take the experience from “this works” to “this feels magical” is not easy, and nothing else in the space has managed to do it.
So ~3 months ago we pivoted to Onyx, the open-source chat UI with:
- (truly) world class chat UX. Usable both by a fresh college grad who grew up with AI and an industry veteran who’s using AI tools for the first time.
- Support for all the common add-ons: RAG, connectors, web search, custom tools, MCP, assistants, deep research.
- RBAC, SSO, permission syncing, easy on-prem hosting to make it work for larger enterprises.
Through building features like deep research and code interpreter that work across model providers, we've learned a ton of non-obvious things about engineering LLMs that have been key to making Onyx work. I'd like to share two that were particularly interesting (happy to discuss more in the comments).
First, context management is one of the most difficult and important things to get right. We’ve found that LLMs really struggle to remember both system prompts and previous user messages in long conversations. Even simple instructions like “ignore sources of type X” in the system prompt are very often ignored. This is exacerbated by multiple tool calls, which can often feed in huge amounts of context. We solved this problem with a “Reminder” prompt—a short 1-3 sentence blurb injected at the end of the user message that describes the non-negotiables that the LLM must abide by. Empirically, LLMs attend most to the very end of the context window, so this placement gives the highest likelihood of adherence.
Second, we’ve needed to build an understanding of the “natural tendencies” of certain models when using tools, and build around them. For example, the GPT family of models are fine-tuned to use a python code interpreter that operates in a Jupyter notebook. Even if told explicitly, it refuses to add `print()` around the last line, since, in Jupyter, this last line is automatically written to stdout. Other models don’t have this strong preference, so we’ve had to design our model-agnostic code interpreter to also automatically `print()` the last bare line.
So far, we’ve had a Fortune 100 team fork Onyx and provide 10k+ employees access to every model within a single interface, and create thousands of use-case specific Assistants for every department, each using the best model for the job. We’ve seen teams operating in sensitive industries completely airgap Onyx w/ locally hosted LLMs to provide a copilot that wouldn’t have been possible otherwise.
If you’d like to try Onyx out, follow https://docs.onyx.app/deployment/getting_started/quickstart to get set up locally w/ Docker in <15 minutes. For our Cloud: https://www.onyx.app/. If there’s anything you'd like to see to make it a no-brainer to replace your ChatGPT Enterprise/Claude Enterprise subscription, we’d love to hear it!
I was told there would be rapid prototyping with AI. Haven't seen any of the above.
Also, open-source works really here, since connectors are a long-tail game. We've tried to make it easy to add connectors (a single python interface), and as a result over half of our connectors are contributed by the community. We expect that percentage to grow over time. This means that compared to something like Agentspace, we'll very likely be connected to all of the key tools at your company (and if we aren't, you can easily add an integration).
It’s nice to see an attempt at an end to end stack (for all that it seems this is “obvious” … there are not that many functional options) but wow we’ve forgotten the basis of making useful products. I’m hoping it gets enough time to bake.
The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left.
In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat.
Something like this could have a nice future as an open source chat framework for building custom UIs if it's well made and modular, but that isn't gonna work well with a SaaS model.
"What's Max's GitHub username?" "I need wire transfer instructions for an incoming wire"
We also index competitors' helpdesks and KB articles to track new features they're rolling out. Our tech support team uses it daily because Freshdesk's AI is terrible and their internal KB search is lackluster. Onyx actually finds things. The value isn't in being "one chat to rule them all" — it's in unified search across disparate systems with citations. That's not getting commoditized anytime soon. Keep up the good work, team.
It does requires having UI components for many different types of interactions (e.g. many ways to collect user input mid-session + display different tools responses like graphs and interactives). With this, people should be able to easily build complex tools/flows on top of that UI, and get a nice, single interface (no siloed tools/swapping) for free. And having this UI be open-source make this easier.
I could just have kept this "negative" thought to myself, but maybe other lurkers think the same. Something for you guys to have in mind. Good luck!
a mobile application that has parity on the same features that ChatGPT and Claude does...
One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise).
Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example).
Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors.
Might have to try this out. OWUI's lagging docs has made managing my own self hosted instance a pain.
PS: Your _See All Connectors_ button on the homepage is 404ing.
"Agents" is a particular area where we feel like we're better than the alternatives (especially if you want something that effectively calls multiple tools in sequence). Curious to hear your thoughts after trying it out!
Onyx Devs: This looks awesome, I will definitely add it to my list of things to try out... close to the top! Thanks, and please keep it cool!
- Instability when self-hosting
- Hard to get in touch with sales when looking for SLA-based contracts
- Cluttered product; Multiple concepts seemingly serving the same purpose (e.g. function calling vs. MCP); Most pre-MCP tools suffer from this
- Trouble integrating it with OIDC
- Bad docs that are mostly LLM generated
Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.
Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.
Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).
Curious, it's a crowded space with other enterprise search companies like Glean and Elastic, and other companies coming in like Notion and Slack.
Why should a prospect choose Onyx over the others?
- "pure chat" experience. From our community (and personal use), we've observed that most queries don't actually involve enterprise search. They much more likely to just require the LLMs internal knowledge (or web search / code execution). Compared to all the companies you've mentioned, we've spent a lot more time refining this more common flow.
- Larger connector suite. As soon as one key source isn't connected, the trustworthiness of the system is dramatically decreased. You second guess "is the info needed to answer this question in there?" for every question. We have a community who builds out connectors for themselves, and then contribute it back for everyone to use. This allows us to cover the long-tail better than companies like Notion and Slack.
- Customizability. An open-source application is the perfect middle ground between a SaaS offering and building blocks. A SaaS option doesn't allow for any customization (we have many customers who have contributed back ux enhancements, small features like guardrails, or enhanced configurations that their users want). Building blocks demand too much domain expertise (search, frontend/UX, ...) for it to be realistic for companies to build something great.
> but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them
I did get a little giggle out of that because I've never heard anyone say that hooking up 3rd party llms to anything was any way secure.
The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly.
Can you call it open source if you need a subscription license to run / edit the code?
In the enterprise:
- permission syncing
- UI-based white labeling
- Advanced RBAC
- Usage analytics UI
Content under backend/ee requires a license, everything else is MIT Expat. Pretty standard stuff.
> Can you call it open source if you need a subscription license to run / edit the code?
MIT is open source, their other stuff isn't. Pretty clear.
Could you elaborate why this approach is confusing?
A common "trick" for commercial open source software is to use a copyleft license, which restricts redistribution as part of commercial products, and to offer a paid license to get around that.
I was responding to parent's question though: "Can you call it open source if you need a subscription license to run / edit the code?"
I'd say no. If you have the code in front of you, it shouldn't require a license to run. Even if the whole point of the open source software is to interact with a proprietary piece of software or service, you could still run it for free, it probably just wouldn't have much utility.
> Many people believe that the spirit of the GNU Project is that you should not charge money for distributing copies of software, or that you should charge as little as possible—just enough to cover the cost. This is a misunderstanding.
> Actually, we encourage people who redistribute free software to charge as much as they wish or can. If a license does not permit users to make copies and sell them, it is a nonfree license.
And even if the source code was only distributed to paying customers, that'd likely be a temporary situation. A relevant quote:
"With free software, users don't have to pay the distribution fee in order to use the software. They can copy the program from a friend who has a copy, or with the help of a friend who has network access."
I do read the GPLv3 such that if someone _does_ buy the code in any fashion, you must provide the source code to them for free. Relevant excerpt from section 6:
"[...] give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge."
But yeah, no obligation to provide the source code for free to non-customers, fair point. Just no ability to stop customers from sharing it with non-customers. Does make sense.
In our opinion, it's a bit silly to build completely in house when you can take something like Onyx as the starting point and be >95% of the way there + have a tons of bells and whistles built in.
We are building a competing open source tool[0] with a very similar focus (strongly relying on interoperable standards like MCP; built for enterprise needs, etc.), though bootstrapping with customers rather than being VC funded. It's nice to see a competitor in the field following similar "OSS Friends" principles, while many of the other ones seem to have strong proprietary tendencies.
(Small heads up: The "view all integrations" button goes to a 404)
1/ large connector suite + good RAG. Answers at scale is hard, and from our enterprise search roots, we've spent a lot of time with it. It's something that many teams expect from their chat UI.
2/ deep research + open-source code interpreter.
3/ simpler UX. LibreChat has a lot of customizability exposed front and center to the user, which is great for the power user but can be overwhelming for someone new to using AI systems.
Can you clarify the license and if this actually meets the definition of Open Source as outlined by the OSI [1] or if this is actually just source available similar to OpenWebUI?
Specifically can / does this run without the /onyx/backend/ee and web/src/app/ee directories which are licensed under a proprietary license?
We have https://github.com/onyx-dot-app/onyx-foss, for a fully MIT licensed version of the repo if you want to be safe about the license/feel freedom to modify every file.
Why do we have to yet again poorly copy an oversimplified UI?
The value of local models comes from their huge amount of settings/control that they offer. Why must we throw that all away?
Yet again, the world waits for good UI/UX for pro/prosumers with AI systems. No one is learning from ComfyUI, Automatic1111, or SillyTavern. No, LM-Studio is not actually prosumer
For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice.
The target customers are enterprises where 70% of the target users have never used an AI system before and have to go through a AI training before being allowed access to the system.
And no one bothered to say anything to them?
I also wouldn't pin it as chat app specific. Quite a few VC funded open core software has adopted that pattern post ~2020(?): cal.com, Dagster, Gitlab
Thanks, lawyers, you make everything better!
As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness. It is open code for sure but it is a business and so incentive is to run it like a business which will conflate with how the project is used by the community.
Btw, there is nothing wrong with that but let's be honest here if you get this funded (perhaps it already is) who are you going to align your mission with - the open source community or shareholders? I don't think you can do both. Especially if a strong competitor comes along that simply deploys the same version of the product. We have seen this story many times before.
Now, this is completely different from let's say Onyx being an enterprise search product where you create a community-driven version. You might say that fundamentally it is the same code but the way it is presented is different. Nobody will think this is open-source but more of "the source is available" if you want to check.
I thought perhaps it will benefit to share this prospective here if it helps at all.
Btw, I hear good things about Onyx and I have heard that some enterprises are already using it - the open-source version.
It's an MIT license. That IS open source.
If they have a commercial strategy - that's a GoodThing. It means they have a viable strategy for staying in business, and keeping the project maintained.
MIT == OpenSource. Pricing == Sustainable. That's a horse worth backing IMO.
At the top level looks like open source but it is not really because parts (the most useful ones) of the project are not. Imagine if python was open source but the core libraries where not. You wont call this open source in the true spirit of open source. You could make the argument that at least it is sustainable because they a have now a business model. It doesn't add up.
I prefer more of a honest take on software. There is nothing wrong to make money while contributing back to the community in some meaningful way or by simply being transparent. In fact this is the best kind and there are plenty of good examples.
All I am saying is that when I see such projects I tend to think that in most cases they are dishonest to themselves or their communities or both.
The source is available and you can do much with it, but the incentive is that this alone should not be enough.
For most people, the chat is the entrypoint to LLMs, and people are growing to expect more and more. So now it might be basic chat, web search, internal RAG, deep research, etc. Very soon, it will be more complex flows kicked off via this interface (e.g. cleaning up a Linear project). The same "chat UI" that is used for basic chat must (imo) support these flows to stay competitive.
On the engineering side, things like Deep Research are quite complex/open-ended, and there can be huge differences in quality between implementations (e.g. ChatGPTs vs Claude). Code interpreter as well (to do it securely) is quite a tricky task.
That being said, I think there is an opportunity for them to discover and serve an important enterprise use case as AI in enterprise hits exponential growth.
On another axis, if you are able to offer BYOK deployments and the customers have huge staff with low usage, it's pretty easy to compete with the big players due to their high per-seat pricing.
The common trend we've seen is that most of these other projects are okay for a true "just send messages to an AI and get responses" use case, but for most things beyond that they fall short / there a lot of paper cuts.
For an individual, this might show up when they try more complex tasks that require multiple tool calls in sequence or when they have a research task to accomplish. For an org, this might show up when trying to manage access to assistants / tools / connected sources.
Our goal is to make sure Onyx is the most advanced and mature option out there. I think we've accomplished that, so if there's anything missing I'd love to hear about it.
I have used vercel for several projects and I'm not tied to it, but would like to understand how onyx is comparable.
Benefits for my use cases for using vercel have been ease of installation, streaming support, model agnosticity, chat persistence and blob support. I definitely don't like the vendor lock in, though.