undefined | Better HN

0 pointssails4y ago0 comments

IMO data engineer roles are further subset into:

1. kafka / streaming oriented software engineering

2. data warehouse and ETL/ELT development for analytics

0 comments

A good data engineer understands and can work with both of these.

They're both "data in, data out" mental models that are part of the Lambda architecture which every data engineer should at least know about [0].

But if you want a specialist streaming person to optimise all the streaming pipelines, then sure hire a specialist.

j / k navigate · click thread line to collapse

0 pointssails4y ago0 comments

IMO data engineer roles are further subset into:

1. kafka / streaming oriented software engineering

2. data warehouse and ETL/ELT development for analytics

A good data engineer understands and can work with both of these.

They're both "data in, data out" mental models that are part of the Lambda architecture which every data engineer should at least know about [0].

But if you want a specialist streaming person to optimise all the streaming pipelines, then sure hire a specialist.

j / k navigate · click thread line to collapse