undefined | Better HN

0 pointscivilized5y ago0 comments

I agree. SQL is nothing to be afraid of, and there's no happier place to be analyzing huge tabular datasets than in a modern columnar database

0 comments

orhmeh095y ago

R’s data.table package is faster at these things out of the box than any single instance of a database server I’ve encountered. This is frustrating because I’m trying to explain some systemic issues we suffer by not using a relational database, but it’s really hard to make my case when data.table is one install.packages away and a version upgrade from Postgres 9 to something a little faster is gatekept by bureaucracy. I’ve been trying for months!

civilizedOP5y ago

You need a columnar database for good performance. Try DuckDB to ease them into it, it's a columnar SQLite.

orhmeh095y ago

Thanks, I’m checking it out, it seems pretty interesting to keep an eye on. Lots of properties that would be useful in our shared computing environment like not requiring root or Docker.

1 more reply

j / k navigate · click thread line to collapse

0 comments

orhmeh095y ago

civilizedOP5y ago

You need a columnar database for good performance. Try DuckDB to ease them into it, it's a columnar SQLite.

orhmeh095y ago

Thanks, I’m checking it out, it seems pretty interesting to keep an eye on. Lots of properties that would be useful in our shared computing environment like not requiring root or Docker.

1 more reply

j / k navigate · click thread line to collapse