As engineers who have worked on data at startups and Amazon, we were frustrated by self-serve BI tools. They seemed dumbed down and they always required us to abandon our local dev tools we know and love (e.g. copilot, git). For us and for everyone we speak to, they end up being a mess.
Based on this, we decided there was a need for engineer-oriented BI and analytics software.
Quary solves these pain points by bringing standard software practices (version control, testing, refactoring, ci/cd, open-source, etc.) to the BI and analytics workflow.
We integrate with many databases, but we’re showcasing our slick Supabase integration, because it: (1) keeps your data safe by running on your machine without data flowing through our servers; and (2) enables you to quickly build an analytics layer on top of your Supabase Postgres instances. Check out our Supabase guide: https://www.quary.dev/docs/quickstart-supabase
What we’re launching today is open source under the Apache 2.0 license. We plan to keep the developer core open source and add paid features like a web platform to easily share data models (per-seat pricing), and an orchestration engine to materialize your data models.
Please try Quary at https://quary.dev and let us know what you think! We're excited to put the power of BI and analytics into the hands of engineers.
Do you anticipate going more towards improving the data modelling capabilities (take on dbt et al) or more towards Business Intelligence (dashboards then hosting then drag&drop query builder all the way until the dreaded pdf export)
Something that is overlooked in the dbt direction is how complex data models get. BI nothing seems overlooked, it is just hard!
I like that you have a clear anti-ICP [dbt customers, analysts]. This keeps you clear of the BI/DWH space. I do wonder how you avoid getting stuck in the BI tar pit [], or avoid getting stuck in the dbt middleware zone. Maybe with a core focus on engineers getting further and further without needing a BI/data team!
[]https://twitter.com/generick_ez/status/1782844341674786952
It's always easier to communicate what you are not, so let's begin there:
- drag and drop query builder - absolute prettiest graphs - tailored to the least sophisticated user
In addition to what I think we are not, I think there is some space for our belief about what the data space is/is not:
- We don't think data self-serve is possible but rather small datasets can be tailored. Fundamentally it comes down to complexity and data is complex. It takes expertise/skill to get value from data.
- It takes experts to get value from data, but as systems get better it will take fewer.
- Businesses should not be data driven, they should be reason driven.
- We don't think data dominates business, it's a supporting tool and so we don't think it will every the entry point for everything but rather supports process, so we want to appear in places where we can support that reasoning, like a chart in a Notion doc.
Now a few bits about what we are:
- Tool for those experts and engineers
- Tool to make them the most productive ever
- Prevent messes that people get themselves in in BI by injecting software engineering practices into the process (we know many companies with full time employees responsible for cleaning up messes)
These are great. I think a lot about BI tools, seldom find any that I would use!
Our thesis is that self-serve is much less important than people think, and we find people often make a mess of never-ending dashboards. Current BI tools struggle to prevent that. We solve this problem with a core of software engineering practices.
I've found self serve to be a really effective tool in getting engagement with BI. My onboarding for new non-tech BI users was always to have them build a basic dashboard for the business process they were most focused on. Maybe set an alert or create a scheduled report delivery. By the end of a 15 or 30 minute onboarding session you'd see the click as they realized what they could do with it.
That mess of never ending dashboards has another name: BI engagement. Though a product can help, having core dashboards and KPIs is a social and analytics leadership problem and not a technical one.
Though I have issues with Looker (their dev experience is crappy), their approach to this is effective: make it difficult for self-serve users to get incorrect or nonsense answers, and make it easy for analytics admins to designate core dashboards and jockey a few hundred custom dashboards and reports as the underlying data models change. Every business unit got pretty attached to what they'd built for themselves.
The problem is that dbt models and BI dashboards are often managed by separate teams. Quary brings the two together, letting engineers define reusable models and build well-structured dashboards on top of them in one cohesive, code-first environment.
There are some core differences that make our product feel quite different:
- Lightdash isn't Lightdash without dbt so you always have that divide even though they have done a fab job of minimizing it.
- The editor for us is in Visual Studio Code which means you don't have that jump and can iterate all together.
- Every thing is version controlled as a file in your repository which means you can add those engineering practices to the dashboards/charts themselves.
"Hate to derail the conversation, but is Quary something I could easily whitelabel to embed BI into my product for my customers? (Passively) looking for solutions in that that don't feel dumbed down."
I appreciate the use of Tailwind scroll-margin on your anchors btw, caring for details is communicative ;)
I've built analytics products, and the good thing about dashboards is that there's budget for them. People like eye-candy, and are willing to pay for it. I like how you picked Postgres as your initial database, because I think it's still the #1 databases for analyics (even though it's OLTP) that no one talks about.
The three products where I think you may want to write short comparison pages are:
- Rill - Preset - Metabase
And I'd take a hard look at ClickHouse as your next database. They're missing a dashboard partner. And I think they're users are much more engineering-centric and therefore a good fit for you than the analytics crowd around Snowflake.
I was just at the Click-house office a few weeks ago - this is a really good idea.
Seems like a cool project!
I would recommend a simpler setup like Metabase Docker (which I re-evaluated recently): https://www.metabase.com/docs/latest/installation-and-operat...
There is nothing to host/provision, so it's simple in that sense. You just run it locally with your credentials and connect directly to your database.
It is definitely not the easiest to set up especially when thinking as a team so we'll keep that in mind.
I've been evaluating evidence and observable framework for a while, and this seems like a nice addition as alternative
But I just realized you require login when using vs code, what is it used for? And can I completely self host this?
Thans!
People will be able to connect their GitHub repositories, deploy dashboards, and share them via our website. The interface will allow switching between branches and time-traveling between different states of the dashboard.
Here's a preview: https://www.youtube.com/watch?v=MD6In-iUd9g
The simple answer is yes: Our focus is code first, from modeling to charts and dashboards, and not self-serve.
We often found that keeping BI applications/dashboards organized is very difficult so we're adding engineering practices.
We love Grafana! It's fab for building dashboards, but it's focused on dashboarding/alerts and on pulling from various data sources, not just SQL.
Quary is purely focused on SQL, and crucially, it allows you to build up and develop more complex transformations.
Dbt makes transformations modular and easier. It applies software development methods to the T of ELT.
- Visual studio code as the editor through and through
- Dashboards are fully defined in code Quary which is different to Rill
- At its core our architecture is also very different, Rill is built on top of Duckdb for that interactivity which can call out to other databases whereas we can call other SQL databases without everything going through DuckDB.
I think here's a few players in this space (dev-friendly BI tool) already: - Holistics.io - Lightdash - Hashboard
These tools all allow analysts to use both/either a local/cloud IDE to write analytics logic, and check in to Git version control.
How do you plan to differentiate with them?
It's not at all clear from the documentation or the onboarding notes how to seed a SQLite in-memory database and the CSVs in the `seeds` directory are sometimes referred to in the sample schemas, but sometimes not. So, kinda got stuck.
I know if I stuck with it (I got impatient), I'd figure it out myself, but it does seem to be a missing element in the docs.
Looks fascinating, though.
Kinda like Elixir LiveBook, but focussed on DBs.
