Ask HN: How to design a data science interview process

Our startup is planning to hire a new employee with expertise in data science. The main responsibility is to work with our existing and incoming data, and extract some meaningful information from it, create prediction and classification models to address relevant questions, and similar work.

For other positions we often have three technical interviews:

    - First one is done by an HR, involves ~10 simple short-answer technical questions, to weed out those with just a fancy CV.
    - second one focuses theoretical skills: e.g. algorithms and data structures, programming language and tools, software engineering, etc.
    - Third one focuses on practical skills: often involves one practical question that the person is supposed to resolve within one to two hours.

We are planning to follow the same pattern. For the third step, we have a good question that can show how the person thinks about the problems, how she approaches them and finds a good solution, etc.

For the first and second though, I'm wondering what questions we should include to assess the candidates? I've read previous HN posts like this one [1] and similar interview question banks, but because I don't have experience in this domain myself, I don't know which questions should be known by someone with good experience in data science, and which one is a bad one ("gotcha" question, something that is often googled, too theoretical, etc.)!

Any advice on how to pick questions for this position?

If it's any relevant, I do have CS background and a fairly good understanding of programming and software engineering. I also follow ML/AI topics out of interest, but have no practical experience there.

Reference: 1. "Data science interview questions with answers" https://news.ycombinator.com/item?id=24460141