Ask HN: How can I transition Data Scientist – Engineer?

1 pointsgmtx7256y ago1 comments

I'm a Data Scientist from an academic background in AI with 4 years experience, working mainly in geospatial analytics, with some basic DL (and a lot more cookie-cutter import sklearn style ML). Lots of spark & bigger datasets. However recently I've started to feel limited by my future options in DS, and have realised I actually enjoy building things more than I enjoy tuning models and playing around with data. I want to transition to more of a dev role perhaps via some kind of ML Engineer position, but I'm not quite sure how.

I think it's definitely doable- i wouldn't want to claim my engineering skills are up to a fully professional standard but they are definitely better than 90% of the other DS people I've worked with. In past roles in small companies I've had more of a wide ranging role and have written APIs to serve model predictions, written tests, worked with CI/CD etc. I think given a couple of months in a dev role I could get myself up to scratch. It's just getting that role that could be difficult. I think there may potentially be opportunities in my current company to transition to a more Data Engineer style position but it would probably mean a big salary cut and the projects in the pipeline don't sound that thrilling. Anyone made a similar transition or wants to share their thoughts?

1 comments

1 comments · 1 top-level

MogwaiAllOnYou6y ago

I have just made this move in my current workplace.

I wanted to do it for the same reasons as you, I enjoy building stuff, and I am of the opinion that data science without engineering is pretty useless as nothing written has the trust of dev teams which is needed for them to plug things into your ML models/pipelines. Some will argue that DS's can do this with dashboards etc, but in my experience they always seem to be made and not used.

In my case, having sat with software eng teams throughout my (short) career, I ended up picking up conversations and challenges they faced as asked questions about technologies and methods they used. I'd go home and look stuff up - enough to understand benefits and disadvantages of things.

Like you, getting to grips with CI/CD, APIs etc etc I had done in past DS projects.

On the money side, I have stayed exactly the same as I was before changing, but obviously ymmv