undefined | Better HN

0 pointswenc2y ago0 comments

No you can definitely use globs in DuckDB.

And no you don’t have to know the exact parquet file. You would treat the Hive partitioned data as a single dataset and DuckDB will scan it automatically. (Partition elimination, predicate pushdown etc all done automatically)

https://duckdb.org/docs/data/partitioning/hive_partitioning

0 comments

cmollis2y ago

ok.. thanks.. I'll try it out. I can think of few use-case that we have where this might be a good alternative to athena.

j / k navigate · click thread line to collapse

0 pointswenc2y ago0 comments

No you can definitely use globs in DuckDB.

https://duckdb.org/docs/data/partitioning/hive_partitioning

0 comments

cmollis2y ago

ok.. thanks.. I'll try it out. I can think of few use-case that we have where this might be a good alternative to athena.

j / k navigate · click thread line to collapse