undefined | Better HN

0 pointsdominotw5y ago0 comments

SQL has taken over the space completely. 90% of data munging and transforms happen via SQL.

I would learn python. Its the number one language outside sql.

0 comments

What do you use if you need to process hundreds of GBs of data?

PySpark or Dask

sql on snowflake

Assuming the data is relatively clean in the first place - there is a lot of data sources where SQL would be an absolute swine to work with.

j / k navigate · click thread line to collapse

0 pointsdominotw5y ago0 comments

SQL has taken over the space completely. 90% of data munging and transforms happen via SQL.

I would learn python. Its the number one language outside sql.

What do you use if you need to process hundreds of GBs of data?

PySpark or Dask

sql on snowflake

Assuming the data is relatively clean in the first place - there is a lot of data sources where SQL would be an absolute swine to work with.

j / k navigate · click thread line to collapse