Ask HN: What is your development and testing workflow for SQL queries?

3 pointstesterhn3y ago2 comments

What is your workflow for developing, testing and versioning SQL queries?

My company has data pipelines that query a number of databases, such as MySQL and BigQuery. The SQL queries are typically read from a file or inlined as a string.

It seems to be challenging to develop, test and reuse SQL queries as you would code in programming languages like Python or Java.

For example, I'd like to decompose queries into reusable units that can be individually tested without heavy-weight external or remote dependencies and complicated setup, have any ideas?

2 comments

chanakya223y ago

Please check dbt package

testerhnOP3y ago

Interesting will have a look, thanks!

j / k navigate · click thread line to collapse

Ask HN: What is your development and testing workflow for SQL queries?

3 pointstesterhn3y ago2 comments

What is your workflow for developing, testing and versioning SQL queries?

My company has data pipelines that query a number of databases, such as MySQL and BigQuery. The SQL queries are typically read from a file or inlined as a string.

It seems to be challenging to develop, test and reuse SQL queries as you would code in programming languages like Python or Java.

For example, I'd like to decompose queries into reusable units that can be individually tested without heavy-weight external or remote dependencies and complicated setup, have any ideas?