Live codings especially are a very good insight in how someone thinks and approaches a problem, never thought about obedience in any aspect.
If someone has contributed to an open source project or has some themselves, talking about those instead is enough for me as well, but that's even more to ask for imo, as not everyone has the time to create a project that is worth talking about and demonstrates their skill.
I can appreciate how some of it offered a glimpse into how I approach problem solving, but the process (not just that aspect) left a sour taste, and over a decade later I'm so glad I didn't wind up in a career position there.
If you want to attract top tier talent, as an employer you should also work to make your interview process fun and challenging, and showcase what sets you apart from you competitors. Too often firms loose sight of the fact that in the interview, they are being evaluated too.
The first candidate had a few small but impressive sample projects in their GitHub, including some API development and other things that surprised me for someone with under a year of experience. I was impressed!
Then the second one came in, who also had a GitHub portfolio with the same sample projects and the same API design (with minor differences). I felt dumb when I realized that those sample "side projects" were actually something that the bootcamp trained them to develop and put into their GitHub profiles. Obvious in hindsight, but remember this was when bootcamps were supposed to be the next educational revolution and everyone was on the "college bad" train.
In a different interview, I mentioned one of my hobbies. The candidate boldly explained that he wrote a certain package for the programming language we used. I asked some more questions before realizing he was literally describing a project I had published! He immediately changed his story and claimed to have misspoken when he realized that we were on to something.
These aren't equivalent to your story, obviously. However, these are some of the reasons why interviewers still want candidates to go through the standardized testing that every candidate receives.
Comparing candidates on equal testing is also pushed very hard on us by HR. If you start judging different candidates based on entirely different standards, people get extremely fussy
- I don’t know how long it took you to do it, and being able to do things in a reasonable amount of time is part of all jobs. Also, I don’t even know if you’re the one who wrote it or if you are passing off somebody else’s work
- it’s unstandardized across interviewees, so how do you compare? Person A wrote a more elegant solution, Person B solved a more challenging problem, and Person C did all their best work on propriety stuff they couldn’t share so they had to share show you a throwaway side project they did in one weekend
I don't have the memory capacity that others do to just stuff leetcode challenge algorithms that I'll never use in my head.
I used to think the same when I started interviewing. Now I'm into the hundreds, and in my experience, that is the correct approach for a peripheral interviewer (not the direct hiring manager), and it's for your benefit.
We have 45 minutes to prove to ourselves, and our managers, and their managers, that you are the right hire. No "feelings" or "hunches" allowed: clear tangible demonstration of competence for the requirements listed, that everyone can read, and be convinced. Where I am, unsure terms like "team fit", "feel" and "think" will get your feedback rejected.
If I spend 30 minutes going over what you did, there are two problems:
1. Figuring out if they actually did the work, or maybe played some small part in a team, takes considerable time. It's not easy, because people are often running with inflated resumes, with the most interesting things usually being the inflation. People can have a good understanding of a project, without doing anything in that project, claiming the whole project for themselves. The path to being convinced that they actually did it ends up being very different for every candidate, and any exaggeration/lie you catch them in makes you question all of it.
2. It only gives a short amount of time to get some deterministic proof that represents the work: typing actual code to solve a problem. If something questionable comes up during then, in the little time left, you're not going to get a chance to prove it's not an issue. If that same issue shows up in anyone else's interview, without me being able to defend you, then that's it.
So, I immediately jump into my "standard" interview. It's easy. An average person will finish on time. A great person will fly through it. It checks all the boxes, gives me the proof, and even expands out into a few things that I know are sometimes questionable for others. It gives a clear yes or no for the job requirements. THEN we go into the projects with the remaining time. If you're competent, you'll have a bunch of time left, but you'll also be a strong "yes" anyways. Any cool stuff you show me will change it from a "yes" to "we need this guy, throw money at him".
If you don't like interviews, you have to blame the liars that make them the way they are. There are so many liars.
What I do to make the live coding challenge a bit nicer:
* I use codesandbox.io (not affiliated) as a live coding environment which is a VSCode editor that executes your code whenever you save. This makes it much more realistic how one would solve it real life. * I have them share their screen and make it clear that they can google whenever needed and that it's expected (who remembers the exact incarnations for reading a file or some API) * I only do one challenge, which most candidates finish in less than 30min. We then talk about refactoring it, building an API design around it etc. * The challenge itself is building a solver for a text based game, which imho is a fun task and is literally reading a file and filtering the content with some ifs.
Any recommendations for improvement are welcome.
I keep my question very light and completely work related. They're allowed to ask as many questions as they want. They don't even have to remember any algorithms (I have a review slide for the relevant ones they might pick). They just have to be able to think and be "seeking" in their mindset, asking questions to fully understand the context, then convert that to code. That's it.
So my simple whiteboard interview question started as "Print the numbers from 1 to 10" but then I had a few candidates that would just write the numbers out on the board.
Since it happened more than once I figured it was a me problem so I became more specific with "Write code to print the numbers from 1 to 10". But then one one candidate would out dumb me and write 10 lines of print statements.
So I thought it would be clever to force a loop of with a really large number: "Write code to print the numbers from 1 to 1 million." but I had one candidate who started to write out a bunch print statements and I had to stop him.
The latest iteration is "Using a for loop to print the numbers from 1 to 1 million putting one number per line." I usually get blank stares. It's either they think it's some sort of trick question or they don't know how to do it. One time I saw an implementation using an if statement to skip 0 and an extra print for the last 1000000.
The solution to that was to tell HR to let me phone interview them for 10 minutes before brining them onsite.
(1) The candidate was a criminal mastermind who somehow managed to fool everyone for 15 years.
(2) The candidate was just super nervous during the interview and experienced a kind of test anxiety.
I've personally been (2) several times.
I got a friend of mine an interview at a BigCo I used to work at. Since he was already a friend, I wasn't allowed to be part of the interview as that might be a conflict of interest, but I did see the review that the interviewer wrote, which said that he didn't know what a for loop was.
Now I had done several projects with this friend, he was someone that I knew was quite competent at coding (and I had seen him write multiple for loops in the process). After he was already declined I did kind of ask about it and he explained that he was asking a clarifying question about how many times something needed to be repeated, they didn't directly answer, so he assumed that that meant "potentially forever" so he did a while loop with a break condition. The interviewer had taken this to mean that he didn't know how to do a for loop.
Now it's possible that my friend was lying to me, but again I had previously seen him write lots of code and I knew he was smart and able to do that, so I'm inclined to believe him.
I'm being a bit facetious of course. For loops (or the equivalent) are still needed in low-level code. But lately I'm very reluctant to use them as there's usually a better design if you step back and think about the problem a little.
Wrote it out, turned and asked how they would like it sorted (they hadn’t specified) since you should almost always have a sort on a query.
The entire room busted up laughing except for the guy that asked the question. He just stood up and left.
Figured that I had pissed off the wrong person. After I was hired, I was paired with him. I later asked him about it over drinks, he said that he walked out because he didn’t need to waste any more time interviewing me, that he went to the manager and said that they better hire me. He was definitively one of the smartest people I’ve ever worked with and a good friend now.
What demonstrates obedience more than having to solve a problem against the expectations of someone holding something of value over your head? "If you don't do as we say in the way we tell you to say it, you don't get this <thing>." No matter you may have a different perspective on solving the problem, even going so far as to dismiss it outright, it's all about the rubric. No subtlety, no interpretation allowed. Make it easier for your interviewer masters.
When people are experienced in a particular skill, they often have reduced inner voice and experience inner quiet as the work just comes to them. Asking people in these circumstances to vocalize their thought process is to take people out of flow conditions into hyper conscious states and perform a metacognitive task, reducing their ability to focus cognitively.
When you ask someone to do something they’ve never done in an interview before, you’re creating a contrived environment that doesn’t exist at all on the job. When I’m creatively working through a problem I often take big breaks after framing the problem so my subconscious can work on it. I often solve hard problems from the prior day first thing in the morning, often with the solution immediately on my mind as I awaken. You can’t do that in a coding interview: if someone doesn’t have the answer in mind readily it’s unlikely the pressured environment is going to encourage its production.
My best advice here is allow them to bring their own laptop and use their preferred editing environment.
If their way of working is to run small pieces of code through the compiler/interpreter to spot errors, they should be able to do that, rather than pointing fingers at a whiteboard and saying "oooh that's not the right way to initialize a std::vector".
If their way of working is to write inside out, outside in, top to bottom, bottom to top, that's all easy in an editor, but not on a whiteboard.
They should be allowed to search for references or even cut-and-paste the boilerplate part of whatever it is, and edit it to complete the task. That is more representative of how people actually code.
Dealing with unfamiliar hardware, let alone a whiteboard, hugely, hugely gets in the way of thinking about coding and is not inclusive of all coding styles.
I would rather be in a hostile deposition because then at least I would understand the opposing party’s motives and intent.
I've had so many interviews that started strong, great conversation, amazing resume, and then they couldn't write a basic function to implement a straightforward algorithm.
Like, they literally could not write a nested for-loop that worked (in a language of their choice!).
IMO, the solution is apprenticeships/probationary offers. "Try before you buy" goes both ways - it gives the candidate and opportunity to try out the company (how many companies like about their culture, tools, quality, and work/life balance?!).
Alas, it requires the industry to shift to support that model, it's very difficult for companies and candidates to support that model on their own without most other companies doing the same.
I have of course met candidates who don't know even the most basic constructs in a language they're supposedly expert in. But none of them were able to "talk the talk", especially when probed directly about problems they've run into and how they've solved them.
I want to be clear I'm not discounting your experiences and I'm also not trying to say you suck at interviewing or anything like that. What I am trying to say is that my own experience tells me there must exist questions that reliably expose this kind of candidate.
I'm idly wondering whether that's why I believe I'm pretty good at interviewing people just by talking with them.
Maybe, many of the people who are capable programmers or software engineers, but who insist that they can't tell anything about a person's ability to code without, say, a Leetcode hazing, are just wired a little differently, and not picking up on all the signals the same way?
I wouldn't think that this is how you would hire a known performer. I can still see the problem, but I can also see how it wouldn't be a problem if it were a more common and more standardized process. And if the applicant were being paid contractor's wages (higher) during the tryout period, not salary-level wages.
There is a massive asymmetry between what a company that can afford to do 1000's of trials at a time to get what they want, and an individual who really can't afford to be culled and lose 2-4 months looking for another position.
Also if you've ever dealt with temp->perm situations, people can change when they go perm. It is a real issue.
---
As an employee: I usually can tell if a company is bad, I don't need to work there 2-3 months to learn that.
So for me there's little to no benefit, and massive downsides in that I don't have ANY real safety net to jump to the next "internship".
Maybe in Europe where they have stronger employee protection, and also better healthcare I'd consider it. But in the US, I'll take every protection you give me.
It's kind of a necessity in places with strong employee protection, otherwise companies would have an even harder time hiring. Now, companies can take a chance on someone without risking very much, so I think it's a good thing. Obviously, the companies have to pay new employees the same as the permanent ones and give them the same benefits, sick pay, PTO, etc.
The flipside is that some unscrupulous companies churn employees every six months to avoid having too many permanent full-time employees. Not common in the IT industry though.
As the new hire, I strongly prefer it for the exact reason you state: it's pretty hard to know if I'm a good fit in a company prior to actually working there. I've used the probationary period twice to get out of such mistakes.
As the hiring company, it's also a great thing for exactly the same reason but from the other side of the equation.
Lots of companies do this. I think more companies should join the bandwagon.
Indeed. I'm gonna vent about my anecdote: we hired someone who talked a good game. Turns out they have worse ADHD than I do, will take a few weeks to do what I expect a couple days, and can't follow a deep technical discussion for more than 10 minutes.
They're trying, I'll grant them that. But their growth curve has been... low logarithmic.
I wish we had been more aggressive with the interview coding tests.
Most companies would love this, especially startups. The problem is that desirable candidates do not love this arrangement - there's an opportunity cost and risk for the candidate here and if you're in demand you can get a solid offer from a company who isn't trying to hedge.
I had an ideal experience with the job that did this with me. It wasn't a probationary offer, they just offered me a seat as a contractor for a month as a tryout. At the end of the first week I found a buggy, unoptimized query that was slowing down a critical part of their site and frustrating them for a while, fixed it, and they made me an offer at the end of the day. Everybody was happy.
I don't know why this isn't done more often.
"Contract to hire" is a popular way of doing that.
Doing a bunch of leetcode problems (something like 20-30 hours over a few weeks) certainly made it easier to answer questions quickly, largely by becoming more fluent in the language and libraries.
When companies use such a problem do they usually include the time and/or space constraints from Leetcode or are they usually fine with O(n^2)?
The problems come when a company overestimates that threshold, either by selecting too hard a task, or believing their work requires more than it does. Then the question selects for (a) savants and (b) people who study for interviews. Unsurprisingly (b)s outnumber (a)s, and then get pissed when they realize how disconnected the material they studied is from the job, and write things like TFA.
if we cant talk about the problem together from first principles and make some progress, and you cant demonstrate that you're tracking the discussion at all then we cant work together on that database.
sure, for devops, who cares.
Describing that as selecting for "obedience" is rather dramatic and provocative, but it's directionally right.
That's what the article is about, it isn't about the sort of basic programming test that anybody should be able to pass without explicit preparation.
But this stems from the article's thesis itself being unclear: both positions are superimposed in TFA, so it's no wonder half the readers came away with one goal post, and half the other.
Motte: Leet-code interviews requiring memorization of problems unlikely to ever be actually encountered during one's career are bad.
Bailey: "[All] Whiteboard interviews are a test of obedience, not intelligence" and the entire "How (Most) Interviews Should Be Conducted" which implicitly rules out any whiteboard/coding questions by not including it.
To the author, if they're here: which position are you actually arguing?
I agree with the motte, as one might guess. If it's truly is the bailey, why should a Senior SWE not be able to implement min()? (… like, realistically, in their sleep?) Perhaps I'm returning false negative in 1/10,000 people, but I'll take that for the true negatives of "cannot, apparently, implement min()". "min()" has pass rates of like 50% in my interviews.
The problem with the suggested interview style is that it does not yield reliable information. People bullshit, people lie. People exaggerate their contributions to something. Worse, the people who you probably do want to hire are going to be modest and honest as to their role.
I'll also add that "whiteboard coding" should ideally not be literal. I really want the candidate to have a good IDE / dev env to work in. I am often hamstrung by what HR/the corp. is willing to provide; if all I've got is a whiteboard, I'm sorry. Everyone being remote should have made this easier, but I still encounter many candidates who struggle to bring an IDE of their choosing in a language they're most comfortable with, to the VC, with advanced notice. (And this is generally a nudge towards "no hire"…)
The answer is “yes”.
what if we could pick the type of interview we wanted? whiteboard coding challenge or work experience based?
Here's a prediction: as it becomes ever easier to fake details remotely, there will be an ever greater emphasis placed on in-person interviews in the hiring process, with the result being that even remote work is less remote: you have to being the physical presence of an interviewer at some point. If the job is in Kalamazoo, most of the people making the final cut will be near Kalamazoo.
The rest is spent co-ordinating, planning, troubleshooting, etc.
If youre going to test me on coding then actually let me code. lol
In the same way a college degree tests your willingness to jump through hoops authority figures set for you, but it also demonstrates a basic level of mental competence.
The interview had nothing to do with the role I would be doing, and worse yet, the interviewer seemed pretty tired and uninterested. So when I was rejected I wasn’t too upset.
If that's your identity, then not being able to solve hard Leetcode problems (which, to my knowledge, are not even actually hard, since they seem to be easier than the problems in actual top competitions like the IOI, ICPC, etc.) is completely unacceptable and thus you will study and practice relentlessly until you manage (if you can); conversely, if that's not your identity, it's probably going to be hard to find the motivation to do so.
Personally, I think the main post hits it right on.
A take-home test, though? They can waste 3 hours of my time to save 5 minutes of their time. Send me a take-home test, I spend a few hours and pass it with flying colours, then they reject me for being "too expensive".
No thanks.
Here is a list if things to NOT say when interviewing for a job:
* I can not talk to you anymore. I need to talk to someone who is competent.
* Are you in drugs?
* Is there a technical person here that I can speak with?
* I need a list of the drugs you have taken in the last 48 hours.
* Are you serious?
* Are you hallucinating? Nobody said that. I think you need help.
* No. This job is not worth being humiliated by someone who is so innumerate that they have to put things back at the grocery store checkout.
I mean, yes that's actually what most large companies want. "Free thinkers" can often be outright terrible employees because they're just not interested in building whatever boring thing a company needs. They put a small number of these people in a box and make the majority of the company build what seem like the most profitable ideas to come out of the box.
Imagine you are interviewing senior levels of engineer, like Guido van Rossum: in an ideal interview, you don’t want to hear him regurgitate textbook problems taken from an exam book.
You want to hear about his vision of where he sees the field in 15 years.
But this is not what companies want.
> Imagine you are interviewing senior levels of engineer, like Guido van Rossum:
The average engineer applicant is not Guido van Rossum. The 99.8% percentile engineer is not Guido van Rossum. Using such a publicly accomplished and prolific person as your example isn’t doing the conversation any justice because it’s not relevant in any way to how actually hiring works.
The key thing I learned is that they were all very talented at what brought them success in their endeavors, but none of them actually made great developers within a company. In two cases, it became immediately obvious that they wanted to treat their job as a side project (that pays the bills) while they further expanded their book/course/open-source empire.
Being the author of popular educational material doesn’t necessarily make someone a great employee. It can actually present a significant distraction, which unfortunately can make it difficult for them to get jobs.
I'm not the OP though; I was informed that the article was shared here, so let me explain some things that I think weren't clear.
Nope. I've never applied to FAANG and never will. And yes, I do know how to revert a binary tree and run zigzag level order traversal from memory using bare C without seg faults, not using JS/Python. But I don't remember other things, and if I have to invest time to train my brain, I better get a good ROI.
I did not call everyone slaves, and I'm not calling anyone a slave, including people working at FAANG. I'm just saying that from the perspective of big corporations, that's how they see us—not just tech corporations, but all corporations.
FAANG companies need to conduct whiteboard interviews because, as I mentioned in the Google section, they have no choice. You're mostly going to be using some tech you've never used before, so they just want to see how you solve problems. I still don't approve of this as it's lacking; it's still a pattern recognition game. I can solve the hardest questions without training, but it would take me time. Expecting me to solve a new riddle my brain hasn't seen before in 45 minutes is just a numbers game. It's pattern recognition, not problem-solving, which needs a more creative aspect.
It's like knowing how to play a song on a piano by memorizing the keys versus composing your own music. Memorizing a song is one thing, but creating your own requires deeper understanding and creativity. Companies need both types of skills, but relying too heavily on memorization leads to a lack of true innovation.
Also, asking veterans how to use a specific function/class/method in library X or framework Y is not it, chief. I often forget most of my own libraries' APIs, let alone other people's. You think I really remember what that method was called? Unless I worked with it recently (within ~6 months), I need a refresher.
However, my problem is with cheap startups and "enterprises" that still use jQuery and Java -1, acting as if they're Google. You're not. You don't make the software they do, you don't pay as much, and working with you sucks.
If you're running a startup and you ask me for take-home assignments and expect me to spend two months relearning patterns I’ve forgotten, you need to pay me as much as FAANG does, or you need to humble yourself and see reality for what it is. Your software sucks, your management sucks, y'all suck. You're not that guy, pal. You're not that guy. If I'm getting a good ROI, why wouldn't I brush up on basic algo manipulation techniques? I'll gladly do it—I'll consider it an investment. If I'm going to be a good slave from your point of view can I at least make good money off it?
People working at humanly unfathomable size organizations being more obedient doesn't sound like an issue to me, nor would it correlate with other qualities, including intelligence or critical thinking.
