Ask HN: How do you guard against ChatGPT use in technical interviews?

We thought that the approach you've outlined would generally be good enough, and has led us to catch instances where people are leaning heavily on LLMs, but our issue now is that everyone appears to be using these things. Admittedly, our sample size here is low (n=3). But it's still frustrating nonetheless.

We've run into a number of people with seemingly-decent resumes (several positions as engineers at reputable albeit non-FAANG companies like insurers or e-commerce firms) who have struggled to complete basic tasks like the Pascal's Triangle question mentioned above.

The intent here is to toss them a couple of softballs that they should be able to knock out of the park, almost like if they were helping a younger sibling with CS 1XX or 2XX level work.

We're not against the use of Copilot, etc. once onboarded. We just want to make sure that these candidates possess basic skills that their resumes would suggest they mastered years ago.

The candidates we're bringing into this screen typically have 1-3 prior positions on their resume, so the point here is to throw them some softballs that they should be able to crank through with some ease to demonstrate that their basic programming skills are there.

We've had experiences where people who have held legitimate programming jobs at F1000 companies struggle greatly with some of the basic questions that I've listed above. I'm not sure why, but it's the case.

We try as best as we can to adjust for anxiousness, I know that programming in front of others can suck, but all the same we're just trying to establish, "before we go forward, can you do some elementary tasks that anyone with your claimed experience should be able to do"

Do you have any suggestions on better questions?

Interesting that you're asking the candidate to have their hands visible. We haven't wanted to have to go that way, but we might.

Do you have any suggestions about the type of questions we could be asking here?

We're basically trying to figure out if they can code generally, or if somehow they've skated by in their last positions without the fundamental skills of programming.

I'm not sure how, but we've come across a number of programmers from F1000 companies that can't seem to hit some of the basics in their chosen language.

LLMs have their place as a tool, but before we empower them with the latest and greatest programming assistance, we want to make sure that they have the skills to do things like critically interpret the output of Copilot, etc.

We want to make sure we're that the people we hire possess the skills they claim, and that they won't serve as a very-slow wrapper for the LLM tools we already pay for.

We'll leave aside the argument of whether or not a carpenter would rather buy a cheap Ikea shelf than build one in some situations, but it's also applicable to this analogy.

Haha we might have to switch to Rust just to ease our interviewing woes here - we're primarily a PHP/Laravel shop, and have tried to be as charitable as possible to candidates by allowing them to program in the language they're strongest in. Perhaps we need to change that.

The screen we're doing is meant to filter out the frauds/imposters and so far has a 100% success rate, but unfortunately we've caught three out of the last three people who have made it to that point. It's become a huge waste of time.

Maybe we just need to curate candidates from OS contributors to prevent this, or something of that nature.