undefined | Better HN

0 pointsmacinjosh8mo ago0 comments

I am working with Databricks' Genies. I have a _very_ complex Enterprise data schema(s). Genies, and from what I can tell, your product work on a set of tables ~20 and expect a well thought out and documented data model.

I have hundreds of tables designed by several different teams. I do have decent documentation on the tables but if I had a nice, organized data model I wouldn't need an AI assistant. If I had a perfect data model my team could write simple SQL queries or give chatgpt a schema dump + a natural language query and it would get the answer most of the time.

IMHO, the big value in this space will be when these tools can wrangle realistic databases.

0 comments

1 comments · 1 top-level

zurfer8mo ago

Well, I can't comment much on Genie, but the core question is always how you scale the complexity.

In Dot, it's divide and conquer. If you have several different teams each of them has to maintain their knowledge base.

A bunch of our customer have less than 10 tables hooked up to Dot, but this data is core to their business and so the analytics agent is really useful. Our most complex setup is on more than 5000 tables, but that was a lot more work to lay out the structure and guidelines.

Also, I don't think all organization are ready for AI. If the data model is a huge mess, data quality is poor and analytics use cases are not mature, it's better to focus on the fundamentals first.

j / k navigate · click thread line to collapse