undefined | Better HN

0 pointscsunbird1y ago0 comments

This is exactly what we do, but with Spark instead. We develop the functions locally in a package and call necessary functions for the job notebooks, and the job notebooks are very minimalistic because of this

0 comments

appplication1y ago

Spark-via-Airflow is also the context we use this, glad of see the pattern also works for you.

j / k navigate · click thread line to collapse

0 pointscsunbird1y ago0 comments

This is exactly what we do, but with Spark instead. We develop the functions locally in a package and call necessary functions for the job notebooks, and the job notebooks are very minimalistic because of this

0 comments

appplication1y ago

Spark-via-Airflow is also the context we use this, glad of see the pattern also works for you.

j / k navigate · click thread line to collapse