undefined | Better HN

0 pointsdustingetz7y ago0 comments

RDBMS = data silo

0 comments

7 comments · 2 top-level

stewbrew7y ago· 5 in thread

You make it sound as if this were a bad thing. RDBMS work well for many use cases. There are plenty of tools around to work with them. Good open source implementations exist.

rqmedes7y ago

The problem space (Business) is not complex, its incredibly simple. Unfortunately the design patterns we use and the tooling is flawed. It has been this way for at least 25 years and the RDBMS, languages, design patterns, and architectures ( 2-tier/ 3-tier) are the cause.

They make it simple to get started and even without knowing what you are doing you can easily churn out something that works if it is simple, doesn't change often, doesn't need to scale and deals with small amounts of data.

"It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy."

RDBMS are the root cause.

jjeaff7y ago

I feel like you are coming from an alternate universe. Nosql is the quick and easy thing to start with, then your needs become more and more complex and nosql just won't cut it anymore. Sure, nosql can scale and perform, but only if your needs are very specific and simple.

There are no major systems out there of even moderate complexity that aren't built on an rdbms.

2 more replies

dustingetzOP7y ago

Consider the case when you are a big company and have hundreds of them

shmooth7y ago

I played with Denodo (data virtualization software) a couple years ago and thought it was pretty legit.

In theory, it could be used to provide that industrial strength abstraction layer between your Tableau/Looker/etc. and your bajillion weird and not-so-weird (RDBMS) data sources.

That would seem to make sense to me from the point of view of -- I would want my data visualization/analytics-type company to be able to concentrate on data visualization/analytics, not building some insane and never-ending data abstraction layer.

The part that surprised me was that Denodo could allegedly do a lot of smart data caching, thus speeding things up (esp hadoop-oriented data sources) and keeping costs down.

I'm guessing the other data virtualization providers can do similar.

kfk7y ago

I have had to work with the Denodo for the past 1+ year, a total nightmare. Data virtualization is a "good in theory" concept but "doesn't work in practice" reality. Going back to the original sources for each query doesn't work, it will always be slower than using a proper analytics data warehouse. Caching doesn't help because at that point you can just do ETL. Also Denodo itself is full of weird behaviors and bugs, my team collectively decided it's worth the most hate of all the "enterprise" tools we use. One thing Denodo is good for is as an "access layer", but then maybe PrestoDB would be worth a shot or maybe even just a sqlalchemy and python.

aceregen7y ago

I don't understand why this gets downvoted. While it may lack context (with the aim of being controversial), it sparked a healthy amount discussion here!

j / k navigate · click thread line to collapse