undefined | Better HN

0 pointssanderjd2y ago0 comments

This is intriguingly dissonant. If your style of interview is "LLM-proof" then why would you care whether people choose to use whatever tools they're comfortable with - including LLMs - if they so desire?

For what it's worth, I think LLMs are very compatible with the style of interview you describe here. For people for whom LLMs have become a part of their workflow, seeing how they interact with them is just one more way to get a sense for how they work and think about things. If it doesn't come up, that's find too! But I don't see any reason not to accept their usage in your interviews.

(But I do think it's key to ask them to screenshare if they're going to go that route, so that you can actually see their interaction to get that signal.)

0 comments

2 comments · 1 top-level

danielvaughn2y ago· 1 in thread

Fair enough, this was a drive-by comment that I didn't give much thought to.

What I meant was that if you're going to give a live-coding interview, I personally wouldn't accept LLM usage. The reason is that with that style of interview, you're essentially validating one of two things - how much knowledge they have on hand, or how they're able to think in the abstract about problems and reliably solve them. I'm more interested in the latter, so I don't care if they use an external reference for trivia. But LLMs simulate the abstract thinking for you, which means I can't evaluate a candidate's ability to reason their way through a problem.

It actually is very important that you're able to think through problems on your own, sans-llm.

sanderjdOP2y ago

> But LLMs simulate the abstract thinking for you

This isn't my experience at all. Or rather, it isn't my experience that people are able to use them to do this, effectively. So if a candidate tries to do that, that's signal.

The point of an interview is to - very imperfectly! - get a sense for how people think and work. If a candidate defers all their thinking to an LLM, that's something I'd like to know about them!

But I've never seen anyone do this in an LLM-allowed interview. Instead, they use it as a more effective version of how people have long used web search and IDE autocomplete, if you allow those. It speeds up the boring parts of interviews where people fumble over silly syntax issues or "what is python's method for xyz thing called" that don't tell me anything about how they think and work in a real setting.

j / k navigate · click thread line to collapse