At least the reason why we have Leetcode questions is because Google did the research and came to the conclusion that those are were good at algorithms ended up being more successful at Google, and THAT'S why we are all suffering through LC. But now that LC has been gamed, I would love to see what the results are as to what makes a successful interview.
Is high performance making economically valuable contributions? If so, there is huge inherent inequality between the economic value of projects, so the metric is worthless for judging engineering talent.
Is high performance getting good performance reviews? In my experience performance reviews are wrought with favoritism, ageism, and probably other subjectivities that create a very low signal to noise ratio, so that metric is worthless.
Is high performance someone who is good at writing software? There are engineers I know who write very good software but lack the decision making or vision to do so in valuable areas. Thus, another metric is worthless.
The reality is there is no general “high performance”. There are smarter and more motivated people, sure. I think testing for general intelligence, personality type, etc., are valuable. But I also think a lot of high performance is about finding someone who is a good fit for the organization in terms of the individual’s experience and skills and the organization’s needs.
- assign low effort, high impact, high visibility, low risk projects to the most favorite engineers.
- assign high effort, low impact, low visibility, high risk projects to the least favorite engineers.
Favoritism in toxic orgs is the reward for "loyalty".
Another way is by reorganizing teams, assigning favorites to a team just before the completion of a project. Or assigning undesirable employees to a failed project.
I've heard that claim times and times again.
Often it's from companies believing in "Best Cost IT talent" or whatever the current buzzword is on LinkedIn. Chances are, they never even worked with an SV caliber dev.
We've all seen engineers that are simply running around in circles around others. Just think of John Carmack. If any of these organizations managed to luck out by hiring someone from, let's say Apple or Google, it would most likely change their beliefs that there's no "high performance".
if they are regularly practiced at a company, this means the company puts enough faith in them to overall promote those who deserve it and to decide peoples' and company's fate based on that. Which then makes it quite a decent interview outcome metric.
If on the other hand performance reviews are as bad and worthless as you claim, they should be stopped completely, and everyone running them should be instantly fired as a bunch of ageist favoritist pigs who are scamming the company.
Google is beginning to show its age, I’ve seen a number of past coworkers get hired there lately that weren’t effective at their job, but I’m sure they crushed some Leetcode.
At this point the whiteboard gauntlet seems more indicative of a lack of critical self evaluation than anything, “we only hire the best because we’re the best rah rah”. Miss me with that shit.
Yup. Goodhart's law strikes again. I am curious as to what the next as-of-yet-ungamed effective hiring metric is.
That's the truth for me, I was shocked some of our worst* interns went on to work at Google after the summer.
* Unproductive, slow, needed lots of hand holding
All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.
I worked for Google and at least back then everyone working there could read the studies showing that the interviews worked. The effect was very clear, with there being no upper bar where the interview score stopped mattering.
If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. But I'm not sure why Google would lie about that, or why they would keep the interview format for so long if it didn't work. The old article talking about Google interviews talked about what they did before they arrived at this format, what they do today is the best interview according to their data.
That's just part of the process, though. They (and many others) have coding questions, system design questions, and behavioral questions. Taken together, they answer the questions of whether you can write code, design/build/contribute to a coherent larger system, and are someone who can be effective at working with others in a company.
Over time, there's been a massive (IMHO) over-index on the coding portion, since it can usually result in an objectively correct or incorrect answer. With massive numbers of applicants, companies really don't lose out on being over-aggressive with cutting people based on these things. As they tighten their hiring belts, I think this will get worse.
Same goes to any subject where a multivariate analysis is almost impossible due to the lack of control over your experiments. I seriously doubt that anybody has any data that could prove anything.
As an interviewer who did this for many companies for a decade, I hire for attitude. I can teach anybody hard skills (software engineering, sql, python, etc.) with enough time but I can't teach attitude.
A person who I can trust because the persistent drive towards goals, can deliver a lot more than a naturally talented programmer with shitty attitude. This is based on my anecdotal data that I accumulated over the years.
Funnily enough this is similar to what the Seal Team figured out about recruiting in a completely different field independently.
If I remember correctly, it was actually slightly negativity correlated with work quality because extremely talented competitive programmers would smash the interviews but write competitive programming style code (aka not enterprise quality).[1]
1) https://catonmat.net/programming-competitions-work-performan...
Isn't the story more complex than that? I certainly didn't just get leetcode questions when I interviewed there, and when I asked, the interviewer said the group hadn't been happy with hires that only knew algorithms.
It's also more common now to have two coding sessions, and even two system design ones, under the rationalization that it reduces interviewer bias.
Google explicitly links to leetcode/hackerrank/etc as suggestions for practice resources in their pre-interview email, implying that practicing artificial DS&A quizzes is now table stakes.
I would be curious as to whether these changes have been data driven, or whether the bar is being increased purely out of necessity to weed out candidates more strictly.
I certainly have seen my share of candidates that do fine in leetcode sessions but then go on to fail in the other sessions for various reasons that allegedly correlate to work performance.
2. Did you get an offer? Did you accept?
Google also operates with their own sets of constraints. What works for them isn't necessarily what works for different types of companies.
Where would we be without such hiring processes that managed to have such set of genius work on Android.
/s
I can tell you that at our company we definitely dodged candidates here and I was very happy with everyone we hired (but you could call that bias, as I was the hiring manager -- but I also worked withe veryone I hired)
FAANG-like hiring has the question "What info do you get from the 5th interview that you didn't already have after the 2nd?" - for the paid trial, what info do you get by Friday that you didn't have by the end of Monday?
[Citation needed]
(Or you are doing what you say others are doing)
> Google did the research
Did they? Where is the proof? Pot, Kettle, black
Note the hiring managers are the ones picking the criteria.
My guess is a dev who passes leetcode but fails to produce is not seen as reflecting upon the hiring manager. It doesn't matter if leetcode correlates or even negatively correlates with the success of the new hire; what would matter is not being able to blame the hiring mgr.
How was it gamed? I wasn't aware of anything. Is this about LLMs being able to solve some of the questions?
Sure you need to understand some basic CS to pass a LC question, but it’s more about recognizing a pattern and having memorized the proper algo for that pattern.
If you were taking an algorithms interview 20 years ago, you’d have to truly know a huge breadth of CS knowledge. But now you can just memorize problems for a few months and never learn the true meaning of “why” a tree is the best data structure in x situation.
It was “gamed” when people learned that you could pass the interviews without truly understanding the material, if you just memorize the rules of the game and can recite them when asked.
Big assumption that performance can be reviewed and scored in some objective way, or if it is that this actually happens. I've seen lots of review processes that pretend to be objective, but absolutely none of them actually were.
Did they? I remember hearing about a research they did and it wasn't so clear cut to its advantages as a "job success predictor"
oral exam style algorithmic interviews are.
Everyones approach looks like it works because if everyone is between a 90-110% worker and the team gets along and works together you will get solid results.
They could have been fooled by randomness, like many "intellectual" types often are.
If the comeback remark is something like, "if you really want this job," I will reply, "if you really want to hire me."
My current job, I told them that I was too busy to do such a thing (and I was), and got hired anyway.
Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know.
Will an attorney give you free legal advice until you decide they are fit to represent you, or will you get free surgery until someone proves they won't completely butcher you? Can you drive a car for free (for 5 - 8 hours) until you decide that's the car you want to buy?
Why is it any different in the software industry? Because we just clack on our keyboards all day and do nothing?
This is a perfect analogy.
I went to a Chevy dealer once, asked if I could test drive a car and they wouldn't even let me take it off the lot. I was allowed to trundle around the rows of cars at walking speed with a salesman in the passenger seat.
I went to a Honda dealer, and they let me take a test drive with a chaperone, but only around a short designated loop of streets "for insurance reasons".
I went to a Mazda dealer, and the salesman said he was busy and tossed me the keys and said have fun.
An acquaintance went to a Subaru dealer, and took the car they were considering home overnight.
You a Chevy.
I think it's more up to the individual dealer than a policy of a given brand
Some years ago, a Nissan dealer offered to let me take a high-end crossover home for the night without any prompting. Ended up buying it. Would buy again from that dealer if I were still in that area.
I agree with your point overall, but I do have to say that it is very common for lawyers to give free consultations with potential clients, often offering very useful advice. This is something I've benefited from more than once, actually.
These are bad analogies because both have extremely extensive tests that are not only unpaid but the tested pays a small fortune for.
If development had the same no one would be asking you.
>I will reply, "if you really want to hire me."
It’s not about hiring you it’s about trying to prevent hiring the wrong person which is extremely expensive in time and money and takes weeks to figure out.
I've passed tech interview challenges only to fail the culture fit because I thought it would "be so easy" after the tech challenges.
And I've passed culture fits only to fail the homework assignments.
In one circumstance, not realizing that a separate recruiter was sending me to a company I had previously interviewed with, I've also seen previous work that I did in a homework assignment, given to me in a different homework assignment with a "what improvements and features would you add to this solution?," when the original homework assignment was the same task. That was a major blow, and gave me feelings they were using some of that work internally.
I get that it's extremely expensive in time and money to hire the wrong person.
On the other end, it can also be extremely expensive in time and money to not be extremely selective of whom you want to work for.
All but one new legal engagement I’ve entered into started with a free consultation. (The only one that didn’t was a straightforward real estate transaction where I knew the lawyer for years beforehand.)
IME you also often get paid (albeit a relatively small amount) for doing them. And FWIW once they want to hire you many employers will actually spend hours trying to sell you on joining. Usually not in the form of an essay, but doing several hours of sell calls or lunch meetings with different people at the company is not that strange (especially if you are higher level).
No, it is because some people have worked at very impressive jobs and probably just clack on keyboards all day and actually did nothing while there, so when you hire them you learn that they actually can't do the job you hired them for. You just can not rely on the CV alone.
In the hiring process there has to be some kind of skill test. If it is not a "homework" then it has to be a whiteboard/live coding/system design type of interview and there are a ton of problems with this type of skill test as well.
We usually give people the choice which route the candidate would like to do. Take a 1 hour interview or a "homework" assignment which took me about 1 hour to solve. Which is probably best because some people really prefer the homework because they get nervous in interviews.
> Will an attorney give you free legal advice until you decide they are fit to represent you, or will you get free surgery until someone proves they won't completely butcher you?
I do not know how attorneys or surgeons are hired, but my guess is that the education side of these jobs lines up much closer what is actually needed to do the job (which isn't really the case with Computer Science) and that when they claim to have done X or Y then it was actually them in court or at the operation table and not someone else from their team.
Clients select their attorney based on reputation of the attorney or the law firm they work for. For Surgeons it is probably similar. If you have the choice you want to go to the hospital with the best reputation. If the company you apply to already knew your name and reputation beforehand then the skill test is probably also not necessary, but that is not usual in my experience.
It also probably helps that both of those jobs require a Professional License to practice these jobs. Maybe if the software industry introduced a Bar Exam then these "homework" assignments would not be needed anymore.
> Can you drive a car for free (for 5 - 8 hours) until you decide that's the car you want to buy?
Usually you can. At least in my experience. Not fo 5-8 hours of course, but long enough to know.
If you know of a better way to hire people: I would be happy to listen.
W/ an interactive session, you get instant feedback (either verbally or via emotional cues) into what they are expecting.
With a homework assignment, it's hard to determine which path to optimize for.
If a homework assignment is necessary, it would be better if that were more of a "probationary employment" type scenario, perhaps at a very reduced amount of pay, to only imply that both parties actually have skin in the game.
Even better, for helping me judge if it's a decent fit? Show me some code you're using in production so that I can code review it on the call. Surely it's not all hyper sensitive.
> In the hiring process there has to be some kind of skill test
Thats a contradiction:
1 - 90% of companies do skill tests
2 - you cannot trust the CV of a candidate even if the company they worked on previously also did a skill test
3 - you think your skill test will enable you to hire the right candidate
So everyone is doing tests and everyone is hiring shit candidates anyway?
That's literally proof that these tests are worthless.
Maybe if the recruiters actually knew what they need from a candidate, companies were clearer about what the job involves, they would stop hiring the wrong people for the wrong job.
There is a large company based in NYC that interviews thousands of developers a year, but only hires a few hundred. Each of those devs do a take home project that takes about 16 hours to complete. The project requires nothing to ask, so this company asks everyone. 30,000 hours is a lot of time - it's about 3.5 years of someone's life - and at least this time is spent each year by this company (and they pay nothing for it), not counting the rest of the interview process, because it is cheap for them to do this.
And we wonder by productivity is down :)
Respect people's time. If it is hard, figure it out. That is your job if you want to hire people and feel good about it.
No there doesn’t. There isn’t one for the CEO is there?
While there are people who are naturally great at interviewing in person or who don't mind grinding leetcode for free for months on end, there is a whole "base of the iceberg" population who a) can't spend months grinding leetcode on end but they can definitely spare a couple of afternoons for one job they are interviewing for – these are normal people with normal jobs and a normal family who don't interview every month just for kicks, they do this a couple of times every 2-3 years, and, b) just cannot code while under stress. You may scoff at b) if you've never experienced it, and while I found it really hard to understand for a while as I also have no issues with getting into deep focus while people are looking and/or trying to talk to me and there's a time limit, it is absolutely real.
Unfortunately, there is an extremely vocal minority (you) who go absolutely ballistic when asked to do a take-home exercise, who absolutely ruin it for everyone else and make hiring managers shit their pants whenever take-home exercises are suggested. I honestly don't understand why there's such an outrage, take-home exercises are the minority already, because there's a strange huge backlash.
Sure, if you're a great communicator (usually native English speaker) and grind leetcode for fun so you can shove 5 interviews in one week, you hate take-home exercises. And that's fine, really, just please apply to the leetcode grinding contests and stop poisoning the well for the rest of us who would like to cater to the large amount of extremely competent engineers that don't fit that persona.
> Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know.
I take it you haven't interviewed for a SE in a while? The status quo requires you to perform months of unpaid labor, not days, just in the form of memorizing the solutions to every single Leetcode Hard problem. How is that better?
I guess because it kinds "scales". Once you are familiar with Leetcode you can use this skill to take interviews with different companies. Take-home exercises dont' "scale".
That being said, I believe people are against take-home exercises exactly for the reason you support it: it's usable code. They worry the companies will exploit candidates by using their code for free. Leetcode is "useless" so it's safe.
That being said the come back makes perfect sense to me. When I was still on the IC track and interviewing, when I was submitted to some "dumb" questions I would ask the same kind to the interviewer. So you just asked me to write this stupid algorithm? How about you whiteboard something for me now?
It was almost always met with dead silence to which I would say "well you're trying to figure out if I'm good, I want to do the same and make sure your team has good engineers".
Of course it never went anywhere since, while they had 55 minutes to grill me I was only given the last 5 for my questions.
Will an attorney give you free legal advice until you
decide they are fit to represent you, or will you get
free surgery until someone proves they won't completely
butcher you?
If you do that in software, you're probably just back on the job market...
I have my own list of questions. If they answer all of them I have a pretty good idea if I'm the guy for the job. The most wonderful part is figuring out if it is an employer is looking for initiative or obedience. If you are running a sheep farm it can be very exciting to see initiative.
I will do take-home assignments (assuming 4~8h of work) or 4h+ onsite only if I am (quite-to-very) interested in the company. This is either the company is famous so I know them well, or there was a good interview process and they passed all of my questions/no red flags. If I'm on the verge of rejecting a company and they ask me for a sudden 4h+ process, sorry but not.
I live in Japan so it's been interesting as there's vastly different thinking companies, you have from the most modern flexible silicon-valley-like company (few, but there are) to very traditional ones that might even be confused when you reject them (again few, but some). Last time I interviewed I told a company I wasn't interested in their offer, only to receive an email later telling me they were not interested in hiring me. I could guess HR marking me as a no-hire was a lot better for that interviewer than marking me as rejecting them, but still made me laugh a bit of how much "no, I am breaking up with you" it sounded like.
I think this is the broken assumption—there are a lot of us who simply are willing to accept a sub-FAANG wage in exchange for a work environment/interview process that we feel respects us.
You signup for a new Amazon job. You are a senior developer you expect to make $400,000 with the stocks/salary. Your base outside of California is 139,000 or 129,000. After year 1 only 5% vests.. after year two 15%.. the average employment length is 1.5 years. So you end up with $140,000/150,000 for working 16 hour days. If you manage to stay 10 years you could retire..(you have to because at this point you hate life) but they don't want people staying at the same level so you need to get a promotion when the 4 year vest up or you will be at your base. Getting one takes the right project and is hard and requires a breakthrough project.
Most people 95% of developers never worked at a faang and those who have, on average worked for 1.5 years. Very few are still employed or seeking faang employment. Faangs make popular entry level position but very difficult to keep for life but if you can survive many years you usually leave the field or create your own startup because of burnout. Faang adjacent companies can be the worst of all worlds same issues worse pay/upside.
I've had interviews with heavy LC and ones where I got plain old fizzbuzz and there wasn't much difference in staff competence or how quickly we delivered.
If anything, the place with the low bar had more well rounded peers I wanted to spend time with after work.
In the world outside HN I very rarely encountered people who'd turn away from any kind of hiring process. Maybe one in fifty candidates?.. I don't have the numbers, but I think I only met such people twice in my life.
I bailed from interviews for different reasons, but I think that homework is a legit way to test someone's skills, so I wouldn't mind that.
The reasons I cut the hiring process short in my job hunts were most commonly:
1. Employer is an MS Windows shop. Sometimes it's hard to figure this out from the job posting.
2. Employer requires employees to use company-provided tools s.a. code editor, or antivirus etc. In other words, an over-reaching IT.
3. Crazy / not very smart / borderline criminal employer. Examples include a guy who had "scrum cards" deck on his desk and essentially showed me to the door when I asked if they used this stuff for real. Another one who couldn't get my homework to run, asked for a Docker image, couldn't run that either, asked for a VM image, couldn't run that either...
----
There's one litmus test I have when interviewing that turned out to be surprisingly precise, and I don't know why. I ask potential employer if they ever use git-merge. If the answer is "no", the company turns out to be intelligent people who are nice to work with, and if the answer is anything else, it turns out to be dysfunctional in more ways than just infra. They will have toxic culture, under-the-carpet skirmishes where each department undermines another department, while at the same time trying to do as little work as possible.
As you can imagine, unfortunately, I had to take jobs where the employer answered "yes" or "sometimes" etc. That's how I know :(
For all I know, I once joined a company where the policy was to only do squash-merges. I left from there at the brink of mental breakdown.
Last 10 years no take home. Today it's coding something in a vc meeting, maybe in a web browser or coding env. Maybe beginners do some coding.
These developers may go their entire careers without ever reversing a binary tree on a whiteboard while juggling two bowling balls on a unicycle.
Well, I personally have always refused to do take-home interviews but happily will do live coding and systems design interviews.
For me it's about respect and power imbalance. A company asking me to do work without them putting in equal effort sets a tone for a culture I personally don't ever want to be a part of.
Like I find take-home interviews disrespectful.
Time-bounded interviews with an interviewer also there (aka FAANG style onsites with 4 hours of interviews) is far and away my preferred process, especially if I can do them all at once. One problem I've seen in a remote friendly world is companies wanting to spread the interviews out over multiple days.
You see people on HN balk at 500k+ engineering jobs even existing, so I think that's your answer.
Or startups. :)
My buddy just got a job at a high-end cocktail bar as a bartender. Part of the interview process was asking him to mix a drink.
Gordon Ramsay has talked about how he'll interview chefs by asking them to make scrambled eggs.
Actors, even famous ones, generally have to 'read' for roles in order to land them.
Musicians interview for seats in symphonies by playing music.
MBAs have to do case studies to land jobs at high end consulting firms.
Hell I applied to Taco Bell as a kid and they made me take a short math test to prove I knew how to make change.
I could give similar examples for dozens of other jobs.
The cases where you don't need to demonstrate some skill in order to get the job generally fall into a few categories:
- There's some outside certifying body like the Bar, CPA, PE, various tradesmen unions, or all the licenses like a CDL.
- The jobs are undifferentiated so the workers are fungible (no special skills required).
- Job skill is immediately apparent (less than two weeks to know for certain if someone can do the job or not).
- The cost of a bad hire is low so you're willing to eat the cost and just cut the workers how don't work out.
With recent layoffs and many talented professionals on the job market, I was compelled to write a blog post about how to build an inclusive hiring culture and find exceptional engineering talent.
If you're involved in your organization's technical hiring process at any stage, I encourage you to give this a read. I share some best practices for conducting effective interviews and improving your own hiring process.
Let me know what you think!
As soon as people figure out the check boxes or the structured pointing system they start to check all the boxes, but it doesn't necessarily speak to the nuances between individuals that make them diverse and both valuable. In fact it can lead to a certain type of person people hired or promote, which can be on good characteristics, but I find many times turns into a "certain type" of person.
I guess what I'm saying is structure can take you so far, but you have to be willing to explore a little bit about what makes a person special, and that many times means not controlling the whole interview, and be willing to have your bias challenged through the candidate directing some of it.
How the heck do you measure actual, repeatable performance? Or skill? Income / promotions / etc is very frequently a horrifically biased metric, for similar reasons to interviews, and we have much larger mountains of evidence showing that to be the case. It seems like there's a pretty good chance these studies are just measuring relative bias between interviewers and the interviewee's management, and concluding interviews are done poorly when they disagree with management. i.e. "structured interviews force people to think more like managers" rather than "structured interviews more accurately measure skill".
Using one bad measuring tool to conclude another tool is bad seems... problematic at best. I will grant that "interviews should measure what managers measure" is often what businesses want in bulk, but that does not seem like a particularly good thing to me.
I think checklist interviews miss the mark as you say. You may not have a perfect rubric, but I’m not grading students on a history exam; I am evaluating them for a role in a given position. In the limited time we have to speak, I want to use my intuition and experience as an interviewer and engineer to rapidly get to where the candidate is strong, and where they may have issues.
I once spent 2 hours coding Tetris for an interview. I lost to another candidate that completed 2 more features than I did in the same time period.
Thats a contradiction in terms. Building something exceptional always involves excluding mediocrity
An inclusive interviewing process does not mean that you hire everyone. It means you reduce the weight of people's biases as part of identifying who you hire (because people turn out to be quite bad at prediction in hiring).
You can exclude mediocrity while also being exclusionary on other axes.
They are unrelated issues. In fact, I’ve even heard of exceptional people being abused/bullied for belonging to the wrong group to the point of being told “you couldn’t have done that” which itself is an assertion of their supposed mediocrity for exclusionary reasons.
Don’t assume you need to be bigoted to exclude mediocrity. Discriminating, yes, but not discriminatory against groups that inclusive hiring policies attempt to protect.
This is wrong by definition
US army was exceptional in 1970's, did you need to graduate from harvard to join? No, they drafted everyone, even your sorry ass didn't want to join.
British industrialisation was exceptional, they didn't exclude anyone, even got children something to do by sending them into coal mines!
Amazon is exceptional, is it hard to become a worker in Amazon warehouse?
Organisation can be exceptional without any individual being exceptional.
Also you could be exceptionally bad!
When you apply for a job as a doctor, they don't make you demonstrate surgical techniques. They take your board certification and ask you questions about what you've done in the past, things that went wrong and what you learned from that experience, and so on. They just assume you have the technical skills if you have the license.
Now, I'll admit that with doctors you are legally required to have the license, and we certainly don't need to go that far. If a small startup wants to take a chance on "unlicensed software engineers" or even offer to pay for the exam as a job perk (like a lot of law firms do for their interns), then great! But I can see a lot of time and effort saved if all the big enterprises would get together and come up with a national certification exam that you take once. Or even better, a series of exams for junior/senior/staff/principle, so neither candidates nor hiring managers have to waste time on tech assessments.
One of the keys would be making the exam inclusive for neurodivergent candidates, people with disabilities, etc. But this can be solved.
Why would this hypothetical certification not have the same problems?
That's when you already worked as a surgeon in a reputable hospital. The same way, the technical screens (leetcode) is generally minimal for hires that have demonstrated a clear pattern of success at reputable companies.
> We need an industry agreed-upon certification exam.
We already mostly do: proper CS/Engineering degrees have a pretty good signal to noise ratio for hiring.
Companies won't share their stats, but they know which schools and programs to target. Real engineering departments typically have job placement rates near 100%.
Why would this not just transfer to the certification process?
When I was working in the “real world” (I’m in consulting now), I could tell the people who just studied for a cert within the first 5 minutes of an interview.
I'd be OK with this if all the answers were just shown to various companies so they could see strengths and weaknesses. If some company doesn't care about DP or low level OO design they can throw out those scores, etc.
I certainly got annoyed in my last job hunt where I had to answer some basic whiteboard easy level LC questions over and over and over. Thank god I got to skip the Google screen. If one more person made me do basic BFS or something I was going to freak out. I understand why they asked it, but I had to keep coding it over and over and over...
I'm an actuary who had to take 10 of these for my profession (pass rates 20%-40% each sitting, takes 4 months to study for each one).
They come with the same complaints about false negatives, unrealistic/random questions, and fairness that technical interviews have.
Anything more complex than that is probably job specific anyways.
At this point, the cargo cult of Leetcode is often more of a hinderance to startups. Everyone wanted to follow Google thinking it’s the best system without asking themselves if they have the same problems or are in need of those kinds of people.
We update and share these points within the team and with the applicants even before they make the first contact with us:
https://stratoflow.com/our-recruitment-and-onboarding-proces...
That means “grinding leetcode and working for a FAANG” (c) r/cscareerquestions.
I am 25+ years in the industry. But if any new college grad asks me for my advice. That’s what I tell them.
I definitely wouldn’t tell them to take a chance of working for any non public company hoping their “equity” may be worth something because they heard that early engineers at Uber struck it rich.
The vast majority of people I’ve hired want to be happy, and I have completely qualitative conclusions from having hired such developers for decades: It works well for everyone.
If you want a job at FAANG badly enough to suffer it, go for it. But don’t ask me validate “that’s what everyone is doing”. It’s not — you’ve been sold a bill of goods, or you’re greedy. Either way, you be you.
You can have pretty good salary in non-FAANGs, make a difference, and match your own software values with what you're doing.
I would even argue that by doing so, you're elevating those company, allowing them to then compete a bit more closer to FAANGs comps, and "stealing your talent" away from those less-matching software values, creating what I believe (from my own values) to be a virtuous circle.
Or you can grind at FAANGs, accept stack ranking and elevated salary, play the yearly promotion game and end up with a very nice pile of money. Then clearly do not expect them to change, and I would argue: you do not get to complain unless you are actively trying to change them from the inside
If inverting a binary tree means swapping the left and right subtrees of every node, I wouldn't want to work with someone who can't do that either and Google is definitely right to reject him.
Rejecting the guy because he cannot do a whiteboard brain teaser is like rejecting LeBron James because he did not make a shot at the arcade basketball game.
I'm not saying the guy would be perfect. Comparing him to LeBron James might not be a great example. Google might have other reasons to reject him.
What I'm trying to say is the current coding interview is a really poor mechanism to gauge a software engineer, especially when it comes to hiring one with real-world engineering experience.
Now does being able to reverse a binary tree mean that you would fail at Google? I have no idea. But we don't really know if that was the reason he was rejected, it's just his own guess. There could have been other reasons.
People like to say this, but in my experience this is not true. It's just that people misunderstand the goal of technical interviews and they often are poor at evaluating their own skills.
First off, these giant tech companies have enormous economic incentives to improve their interview processes as much as possible. They also do a pretty rigorous assessment of the effectiveness of their interview process (Google, for example, has publicized some of their data). I'm not saying these tech companies interview processes are perfect, but I also have a problem believing they're so fundamentally flawed that these companies can't figure out how to fix them given the giant economic returns they get for optimizing their hiring processes.
Moreover, as some other comments mentioned, many companies (and individuals, myself included) believe it is much worse to hire someone who ends up not cutting it, than missing out on a potentially good hire. I can list out all the reasons why, but Joel Spoelsky has a pretty famous essay from a couple decades ago on the topic that explains it well [1].
Thus, it's not surprising hearing a lot people complain that they can do the job, but they aren't good at interviews. Because, from Google's/Microsoft's/etc. perspective, they're fine with a bit higher false negative rate if they can greatly reduce their false positive rate. And my experience matches that: I have never seen a candidate who did awesome in "whiteboard-style programming questions" who couldn't cut it programming-wise (they may have had other issues, but "coding productivity" wasn't one of them). Now, I certainly believe and have seen that there are some people who aren't good at these questions who can do a job well, but there are also a ton more people who can't do the job if they can't pass a technical screen, so hiring any of these folks means much more risk.
I also think that whiteboard-style coding questions help show a quality that is very important to businesses, even if those questions don't represent "real world" work. There are basically 2 types of people that do well at these questions: people who are just naturally smart and have a ton of experience to the point that they wouldn't even need to study to do well, and people who are of more "normal" intelligence/ability, but who can do well if they study a ton. Either of those two groups would likely do well in a programming role. So often I hear the complaint "I'm a busy person, I've got outside responsibilities, you can't expect me to spend all this time studying". And that may be true, but you'll be competing against people who are willing to study, so I don't think you can fault Google et al for favoring people who show a willingness to do more preparation.
1. https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... "And in the middle, you have a large number of “maybes” who seem like they might just be able to contribute something. The trick is telling the difference between the superstars and the maybes, because the secret is that you don’t want to hire any of the maybes. Ever."
I would rather hire an engineer with a strong business or user sense - reading between the lines of requests and anticipating future issues or uses adds so much more value in a real sense.
To me, these are great entry level questions because it is a good baseline for new grads when you have little work experience to judge. Past that, it is like making a lawyer take a mini bar exam for every new job - a waste of effort if you want to hire for specific skills and experience.
I went out of my way to avoid homebrew (still do) when I worked at google because it would reliably fail to complete some key operations in a dag, hence the interest in ensuring developers know how to do CS things.
How complex is homebrew ? Can no one else replicate it ? Why should a company hire for something you did that's simple ?
What are the skills he posses that no one else has ?
Learn your basica dsa stuff for gods sake people.
Yes, yes, 1000x yes. I was recently rejected after two technical interviews from a company that my former principal engineer that I worked directly under had referred me to. The position was to work under them again, which is why they referred me. The feedback I got from the recruiter was that it wasn’t the result they had expected, and I hadn’t achieved a specific number on their technical assessment. My understanding of this after some discussion was that the lower score was due to my speed in answering the leetcode style questions in the live interview with another engineer.
Here’s the secret I never brought up with the company while interviewing: In high, school, college, and for the CPA exam, I received accommodations for extended time and testing in isolation to reduce distractions from my ADHD. With those accommodations bringing me up to an even playing field with a neurotypical test taker, I was able to get into a good university, graduate with a bachelors and masters at the top of my class, and pass all four sections of the CPA exam on my first attempt. In the real working world, I have never needed extended time. I always deliver what is asked of me on time while I have witnessed neurotypicals show up to meetings with their work majorly behind.
I have always hesitated to bring this up with companies because I fear they will make the incorrect assumption that extended time on testing implicates that I will be a slower worker, which I have not found to be the case. I don’t want to introduce any biases for the interviewer to pick up. For whatever reason, testing with pressures absolutely slows down my thinking. In the real world, I have found when I face particularly tough problems, I find solutions after going on a 15 minute walk outside or while taking a shower in the morning. You cannot test for that style of problem solving in these high intensity algorithm technical interviews.
I certainly miss having a CPA license as evidence that I was a competent individual from my previous accounting career, which allowed all parties to skip technical questions in the interview and instead focus on fit for both sides. The software engineering industry suffers from too great of an emphasis on absolute performance levels in my opinion. To pass a section of the CPA exam, one needs to score a minimum of 75. What do you call an accountant that passed every section of the CPA exam with 75s? A CPA.
Meta recently sent me their interview prep material and it says:
> In your tech screen, you'll be asked to solve one or two problems in under 35 minutes. Practice coding solutions to medium and hard problems in less than 15 minutes each to help you be ready for the constraints during the interview.
Less than 15 minutes?! The only way I can solve them in less than 15 minutes is if I've seen them before.
I think that's the point...
Now even if you have a living and respectable proof that you have worked with this person and was good, it means almost nothing. Let’s five rounds of interviews. I have seen the same pattern even for internal transfers! Crazy!
As a hiring manager, when I get a reference from a good report I tell HR to CT the BS and trim down the process. I talk to the candidate, have tem meet a few folks from the team and we're good to go.
As a candidate, when I come in as a referred candidate and you throw your 2 hour take home in my face I'm walking away.
At this point, being referred just means that you have a good chance that your resume is at least passing the first review, but that's pretty much it.
I've been in the industry for ten years, but if I had to narrow down to the folks I'm not currently working with who worked closely with me, it's not that many people, unless we're going to people from 5+ years ago, and at that point, if I was on the receiving end of that, I don't know if I'd trust a reference that was that out-of-date.
It’s hard for me to tell which companies require those style of interviews, but I’m not working in the Bay Area and have found when applying to positions remotely there, they seem to have a much greater emphasis on leetcode than companies in my area.
It's funny how they claim that a bad hire is devastating, and they can't rid of them easily, but somehow they can do mass layoffs and get rid of a bunch of engineers easily.
Yes, you are expected to hit-the-ground running on day one, but no one will immediately operate at their full potential. Even with all the shared best practices in the world, the secret sauce is the part you have to learn.
As an employer it's very hard to know if the reason for someone's uneven performance is due to ramp-up or if they are just not a good fit. Without a rigorous interview process, so many months would be wasted waiting to get a clear signal on that person.
That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees.
Another implied reason, good engineers want to surround themselves with other good engineers. So knowing its hard to get into a company signals to each applicant that the other employees there made it through that process.
Maybe that's because companies tend to hire whiteboard-master generalists rather than subject-matter specialists who may not be great at standardized technical interviews. ;-)
Also, if the company culture is ultra-bureaucratic, maybe the company should fix that instead of wasting months on every new hire.
Seriously, if a new engineer can't commit code within the first week, that's a company problem, not an engineer problem. Of course their code shouldn't go directly into production, but that's true of any new code. Give them something small to start, like some bugs to fix.
> That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees.
Technical interviews can't determine this.
> knowing its hard to get into a company signals to each applicant that the other employees there made it through that process.
I realize that's a signal, but it's not necessarily a good or accurate signal. I think it's mostly PR and hype. Reminds me a lot of fraternity hazing. Google engineers believe they're the best, and some of them may be, but some of them don't impress me at all. And as I mentioned, engineers tend to move from company to company anyway, so if Google engineers are "the best", they're constantly losing the best too.
I hope you realized that this should answer your own questions. Layoffs may be (relatively) easy, but firing someone for "you're just not cutting it" is much, much, much more difficult.
First off, most companies are loath to do large scale layoffs unless there are strong economic reasons to do so - many of the FAANGs have never had layoffs as big as the recent ones. So if your only chance to get rid of bad hires is every 5-10 years or so when there's an economic downturn, that's a problem.
But more importantly, while it's generally straightforward to fire someone who's flat out bad (as there is usually plenty of data to emphasize why they're bad), firing someone for cause who is just kinda mediocre is nearly impossible in the tech world in my experience. For example, if someone can do the job, but say is 50% slower than your average programmer (I've definitely seen this), it can be extremely difficult to gather enough evidence to fire that person. And it usually sucks for everyone involved, because often times these people who are slow are hard workers, but they're just not as capable as their peers.
One of the reasons you see the behaviors you see in technical interviews is precisely because hiring a kinda-OK-but-at-or-slightly-below-par is basically the worst kind of hire you can make.
It's actually not. When upper management is motivated to fire people, they get fired fast. Whether that's an individual person or a large group of people. We've seen this happen over and over. Self-imposed bureaucracy is the only thing that prevents fast firing.
> it can be extremely difficult to gather enough evidence to fire that person.
You don't need evidence. There's no such legal requirement. It's at-will employment.
And I don't want to hear about potential lawsuits. These are ghost stories, designed to scare, but ghosts don't exist. Show me the lawsuits. Incompetent people who are suddenly out of a job don't have the time or money to file frivolous lawsuits (which could get them blacklisted from the entire industry). The ratio of lawsuits to firings is close enough to zero to be negligible, and certainly big tech companies can afford to defend themselves.
I've worked with 200+ engineers and I know of exactly four that were fired for performance. But probably another 40 were quite bad and we would have been better off without them, they just didn't exactly meet the bar for 'so bad we have to fire them immediately'.
> the costs of decreased productivity and morale
I don't dispute any of that. I just mean that they can legally do it, and they don't have to justify it, they don't have to put employees on PIP, they don't have to give reasons why every employee was included. I mean, Elon Musk can basically walk into Twitter and haphazardly fire a ton of people. The consequences may be bad, but it's "easy" in the sense that he can just do it whenever he wants. Even more so for individual firings as opposed to mass layoffs.
You want to see repeatable behavior and a general interest in going through the process. If someone takes the time to apply with homework and is able to articulate well, it gives you so much valuable signal.
Pressure cooker style interviews only reveal someone can remain focused under stress and that they studied their leetcodes.
Been treated in past like I'm avoiding the "important part" of solving their quiz and that it's some softball topic.
I can't think of any reason why anyone would ever do this. Just navigate the tree in the reverse of your normal direction instead.
Why not ask the much more interesting and potentially useful question of balancing a binary tree? Or do something else recursive, if that's what you're after.
Or even just when would you use a binary tree? Figuring out which data structure is appropriate for the problem at hand is the hard part, how to implement operations on the data structure is easy in comparison, you can just Google it.
Every (recently with raft protocol) multi master distributed system out there interacts with Zookeeper for assigning leader and maintain configuration.
Do you know how the syntax or api calls look like ? They are node path in a tree. You want to store something ? That's a tree path again. You want to listen to some change ? It's tree path again.
As I said, most people are ignorant here and don't do "true" computer science engineering in daily life.
Most devs simply convert business logic into bunch of apis + adhoc implementation.
Did you ever work at a banking firm ? I've read their codebase - they are structured as trees, every damn thing is tree. It's a headache to navigate, code, heck even the objects are literal trees.
As I said, people are ignorant and think world revolves around them.
Honestly? Because that's harder.
The swapping question is basically a softball / FizzBuzz-style question to test the most basic familiarity with data structures, pointers/references, and recursion.
Also, stop giving take home projects. Bad candidates will cheat them and good candidates will not even do them. If one of the random startup names listed on the author's site sent me a 12 hour take home project I would delete the email. Do you think they pay twice as much as the bigger company that only makes you waste 6 hours doing a whiteboard? I doubt it.
And I think that smaller companies copy this as a part of the tendency to copy large companies without thinking about whether the thing they are copying actually makes sense at their scale. In this case it can be very damaging, because false negative for a startup with a limited pipeline can be very bad.
You're right about the copycat behavior. This goes all the way to top of funnel: these small, even trivial-scale web application startups just don't have hard engineering problems. Many imagine they do, or imagine they will once they take off, but the work they're offering these high powered candidates they claim to want to hire is like, wiring up CRUD apps and making javascript buttons. It's not technically deep work, it's product work. A little humility about whether or not your tech startup is truly doing "tech" problems would, I think, fix some of the expectation/reality mismatch people are having when they complain about how hard it is to hire engineers.
And sure, lots of people join Google to work on world-scale problems and end up wiring CRUD stuff anyway. But they can at least plausibly offer some technical depth (or could anyway, perhaps Google's reputation as a great place to develop an engineering career has been fading).
(this is not to knock on "CRUD" but to highlight that a technical problem solver is an overlapping but not identical skillset to someone who can work with a fast moving team to quickly and reliably develop product changes)
You the candidate, did not get the job because they did not like you ( e.g. you had a voice similar to the kid in school who was bully to your interviewer etc.). For most software positions out there, a relatively mediocre level of skills is sufficient. No fucking need to hair split on a person's technical skill.
If you truly care about the candidate first judge him on the 'cultural' fit. If he has crossed that barrier then the following advice from the article is a great one:
>Give candidates a heads-up about the attributes or topics the interview will cover and any other information you can reasonably share upfront.
All other advise in the article like paid assignment etc. are also great.
I would say it's more common to not get hired because you didn't have a strong yes. Maybe everyone thought you were ok but none of the interviews really wanted to argue for your case. There are also cases of people that are just obviously not qualified, like every single interviewer says 'no hire'.
If you have two almost identical candidates, but the other has better credentials, it’s difficult to turn them down - if you have some HR guidelines to follow
I assume you mean empathetic. Same word is spelled “Emphethatic” later. (I tried finding a way to reach you privately, but your site “about” says you have contact methods on the left but, on mobile, there is no left…so here will have to do.)
Why can’t anybody trust verifiable info these days - that you worked for $COMPANY doing some $JOB for $YEARS. Why are resumes completely thrown out the window and you start from ground zero on a whiteboard when you have over a decade of experience. I wasn’t practicing leet for the last decade, I was doing actual engineering.
I work in aerospace which often does a STAR behavioral interview (very light to zero technical interview) and I can honestly say that I work with some of the brightest people I’ve ever met.
So it's not that I don't trust that they didn't work there, but that doesn't mean that they can do a good job here. Every company has low performers, perhaps their previous employer was just bad at detecting them.
0. 45 minute homework/prescreen. Provide an (optional) pre-setup environment so it's mostly about coding and not about building/installing deps.
1. on-site where you chat about your solution, mostly an ice breaker/introduction to the team.
2. pair-programming to extend the homework or work on a simplified but real problem encountered day to day, open book
3. design review
4. code review
5. behavioral / case study
All of these can be pretty objective and don't rely on any memorization. All this should be pre-canned so individual proctors don't come up with their own questions and you're comparing candidates around the same prompts. It's amazing how few companies even manage these basic steps. I think most importantly the hiring should be done by a committee of actual practicing engineers - that means if you have checked in code in six months you aren't a vote on the committee.
1, 2 and 5 should be more than enough. You get to talk to them about their past expertise and even combine it with some design discussion, you get a pair programming session and a final casual discussion. Why do you need everything else?
This didn't make any sense to me, so I looked it up, and it seems exactly as senseless as I had thought. Is the 'inversion' not the same topology as the input?
my solution: "auto invert(regular_tree x){return static-cast<inverted_tree>(x);}".
How many countless stories like that exist?
For example, people messaging for a chat "found your work on project X and saw your github account and found about your past projects, we are looking for some one like you", then on the day "oh sorry to let you know last minute but X and Y happened".
Bunch of time wasters! This is the reality!
Once in the job, it's funny to see who actually does the work. Zero contributions for days, etc.
There are a lot of people out there handling these processes and they are bad, really bad!
There's the rub. Women are disproportionately likely to spend time caring for children and have less free time available for hobbies. Personal projects might be a positive indicator that someone really cares about computing but the projects I've been praised for in interviews make up ~1/500th of the time I've spent developing my skills. It can objectively display the quality of code that a candidate can write but its absence isn't a guarantee that the skills don't exist.
HR and recruiting relationships are sparse at best.
It’s working well, and has avoided at least two false negatives since implementation within the last six months.
There is a large diversity in how developers are effective. When you force people into one funnel you lose the rest of the ecosystem. Meet people where they are, the only metric that should matter is effectiveness
If you still want "exceptional talent", but not algorithmic interviews, then you end up biasing towards white guys who have a ton of projects to show you.
Actually, I think this should be verifiable. Select some companies that we think have exceptionally high bar (you could use compensation as a proxy, acknowledging it's imperfect). Then classify them based on whether they do "leetcode" interviews or not, and check their diversity reports. My bet would be that the "leetcode" companies do significantly better.
* Caveat is that companies people think do "leetcode" actually usually ban questions that appear on leetcode.
Not really—if I don't have time to do an hour-long take-home assignment, what makes you think I have time to practice leetcode-style questions? The take-home assignment is usually testing skills that I actually use in my job on a regular basis, so I don't need extra preparation, I just need a block of time to sit down and do it.
I agree that expecting people to be able to show side projects is a mistake, though I'm not sure why you think that the bias there would be racial—I would imagine it would be much more a filter that excludes people with families and/or non-computer hobbies.
So I'd add another criteria: interviewers need to be trained!
Plural: Criteria
SCNR
When interviewing someone, I would still want someone to work through a problem live with me to see how they solve some problem. I don't care about the end result, I want to see how you're getting there.
There wasn't growth in that team due to mediocre hiring and eventually all the good ones - left to other companies.
My current team is an infra platform and has lot of growth as IC. Everyone is learning something in-depth and are explorers - rather than blind sheep. The bar here is higher than the one for my previous team.
Our team requires you to know about whatever you talk on, not just usage but it's internals - why ? That's what we do daily. It can be about scheduler, checkpointing, auto scaling, concurrency, different data structures & algos, integrating with ecosystem, etc.
Even soft skills - like helping others, taking feedback, communicating clearly, etc.
Yeah so, mediocre will always be a burden to team.
Lot of my thinking is based on visuals and emotions -- It's challenging for me to transcribe to English on demand and it interrupts my process -- it's somewhat like painting.
I always shine on take-homes since I'm allowed to be my authentic self. I'm enabled and have the full capacity to do my rituals, routines, and quirks.
Admittedly, this means I won't succeed in cooperative environments like pair programming. I'm better off left to my own devices.
Whatever happened to getting to know a candidates work? How about look into the work that a person has done and take the time to understand where a person's skills are. The problem with tech hiring is we have people trying to cut corners. So-called 'non-technical' recruiters doing interviews with a checklist, companies that treat people like hoop-jumping monkeys, and generally f*king idiots that won't do their job (they get paid for it, why again? They're not actually doing their job.)
Hiring is not a complex problem. The problem is literally incompetent people doing hiring.
Game changes if you actively contacted someone.. if you're no BS, assumption is you know who you contacted and why, hence only thing to do, once contact established is not to oversell and pay well.
Pay-well can constitue compensation as well as time.
It’s more conversational, and you don’t have to live in hypotheticals.
We all know that skilled engineers will learn whatever skills they need to on the job, so less and less am I interested in what they can do in the interview pressure cooker.
It's a bit orthogonal to the concerns in this article, but in some ways it's much more important.
What I wonder about is given an org that is able and willing to compensate at market clearing rates, how do they get the word out well enough to get engineers interested. Because the other big BS in hiring is the whole recruiting side of things.
It's so blatantly obvious that interviewers are basically trying to hire themselves, and will almost always select candidates whom they share the most personality traits with.
Also, I see the hiring process as similar to wanting to be a politician ie anyone who really wants to be one and is just really good at it should never be given the job.
The people who impress you the most almost surely have simply put a ton more effort into gaming the process with long leetcode sessions, live interview practices, and other bullshit tricks to convince you that they are the best person to hire.
With so much as stake, why wouldn't young devs spend tons of time working on interviewing skills and not really giving a damn about developing the real skills needed to be a goto resource at Big Software?
It's very much like taking steroids in professional sports...well no shit your taking PEDs when your career paths are either making generational wealth fucking with a ball or working selling Jordans at the local shoe store.
If I could give one minor persuasive writing critique to the author of this article though, I'd suggest not emphasizing inclusivity (which is basically bog standard, meaningless corporate drivel by now), but emphasizing that changing the typical interview pattern ensures that you're casting the widest possible net for talent. There's business people out there that couldn't give half an ounce of care for doing a solid for whatever the heck they might think a neurodivergent is, but if you emphatically frame this a bit different (solely as the company missing out on talent) I think the argument instantly becomes a lot more appealing to a pretty large group in the business world.
1) Doing a relatively shallow but wide survey of the technology I'll expect them to be responsible for. Because we use k8s, I steal the old "type google.com into your browser" question and make it, "I type 'kubectl get pods' and hit enter. What happens to make the list of pods show up on my screen?". From there, you can dive into basically any part of the stack you want.
I'll often ask them to explain to me what the difference between a container and a VM is, as though I were an intern, and then I'll ask probing questions about things they get wrong or things they leave out that I think are relevant.
This isn't to pass/fail them, necessarily (though some people have done so badly that they essentially failed themselves), but it's to see where their familiarity and comfort level is with the tech at hand.
2) talking through their resume with them, and doing a deep-dive into a couple aspects of their recent history - why did they do a thing? What alternatives did they explore? What was the reason they went with what they did? How did they implement it? What problems happened? Who did they collaborate with and what was the precise scope of their involvement? How did they measure success, and what was the follow-up?
I don't expect anyone coming in to have a deep knowledge of the tech stack we have. I do expect someone to have deep knowledge of the technology that they put on their resume, though.
My hit / miss rate is pretty decent. There have been a couple of times that I said no when I should have said yes, but I'm okay with that ratio.
1. If there are challenges, particularly if they are take home tests, it is important to make these reflect the sort of work someone will do without raising concerns that the work will be used by the company without pay. Candidates will spend time on relevant challenges and be happy. They will not be happy about irrelevant challenges. And interviews go both ways.
2. Dispense with "good questions" and go instead with "what do I want to know about a candidate.
3. Ask yourself before you start hiring, "What makes those who are successful at this company successful?" And from there, start building your interview structure.
Not every company will be the same, or will be good matches for the same candidates. The key should be to figure out what you need and use the interview to determine if the candidate actually is a good fit.
Unfortunately this cannot have data because it relies on a bunch of human judgment calls.
My own interviews have a list of topics I want to cover (non functional requirements, data experience, app design, infrastructure, etc), so I guess there is some structure. But I mostly run the interview based on their own experiences and projects they have worked on. So we will focus on applications and systems they have worked with in the past. And then I see how deep down those rabbit holes of their own system they can go.
They have different values. Different expectations of what is normal or important. "Culture", "team fit" and other bs.
Everything changes. New people come who don't know the past.
It's kind of a "choose your own adventure" style interview.
I explain upfront what the process will be, that during the interview "I don't know" is a preferred answer than BS.
Then I start with this question:
"Your task is to take some data in from a user, store it, and then present it back to the user. How do you do it?"
Based on their answer and follow up questions, I follow them down the path of their preferred stack.
This is rare, but I love it when prior to answering, a candidate asks me clarifing questions, like "how many concurrent users will the system need to support? What sort of performance is necessary?" Etc...
Many just assume that I'm asking about web development, so if they go down that path I challenge their assumptions, and follow up asking about their reasoning.
If they pick a framework, I ask what the advantages and disadvantages there are to that framework and how it compares to others.
Same with databases, etc..
It becomes pretty clear quickly what sort of level they're at. Many times the answer on a framework question is "that's what I used at my last job". That's not necessarily a poor answer, but it is informative.
I try hard to make it a casual conversation, like the sort you'd have with a technical stranger at a bar or something, though I understand that's impossible given the power dynamics and stakes involved. Still, I try.
So far, it seems to work pretty well for me. I have the luxury of keeping my team small, so I don't have to come up with a scalable "one size fits all" process, I'm able to keep it personal and relevant to the candidate and their experience.
I'd pick someone who really clicks with the team and is smart any day over someone who's brilliant but hard to work with.
Other things that are actually useful to consider (others have mentioned these too).
1) Your company culture is important for you to know; for you to codify and for you to communicate to your candidate.
2) In some companies, you will have tonnes of unsuitable applicants because of your brand, you should not optimise for people that aren't suitable - filter them asap
3) Your whole onboarding is a lot wider than just an interview. For many of us, we should be asking where/how we would expect to find our candidates and optimise those places. Do we visit hackathons? Is our recruitment page(s) clear and does it articulate what we want?
4) Your entire process needs to be like an interative development. Did you hire a bad person? What could you change to catch that in an interview? Wrong technical questions? Not enough about comms or culture?
5) In many cases, a company will hire someone that comes across well even if they don't necessarily tick the boxes so don't assume that you didn't get the job because you failed the whiteboard test, maybe you weren't as good as you thought?
6) Candidates need to think carefully about their approach to interviews, I would say more candidates than not seem almost entirely unprepared for normal interview questions and perhaps expect their ambience will get them the job! Study, swot up, never used owasp for web apps? Don't say that in an interview, spend 10 minutes learning about the top 10 and answer confidently.
7) Recruiters are in it for large commissions. They kind of care only in-as-much as getting a good reputation might help them but everything is second to money so don't expect them to place you well, don't expect them to sell you properly and if you get rejected, don't expect them to call you!
Yes they know its a 100% a trap, but good engineers won't be able to resist responding on the off chance it is real.
Don't click this... it is clearly a trick... you already knew they never have overstock sales. yet had to click this anyways... =)
"Process is broken" continues to be the theme where no parties agree whether a basic algo or leetcode or takehome is sufficient yet continuously reject professional designation.
Folks, keep in mind that at the end of the day, you are hiring a person, not an object with a bunch of methods.
Depending on these, you end up with different attitude and questions.
I quite like how things work currently. It's easy to tell the difference.
Being "good at the job" is actually bad because it reduces the team's required size.
Almost all the incentives revolve around this. Not only compensation, but entire hierarchies that revolve around EB1C dangling. (managers and executives at multinationals can get a green card faster)
You can't do that with a take-home (and I'm against take home as the signal to noise ratio is too low) because people will cheat and have them done by someone else.
I've heard horror story of a "senior" engineer from "his country's top school" being interviewed for a technical position by several non-technical managers and HR reps. They only included an engineer in the final round, which was basically supposed to be rubberstamped anyways. He was then asked to implement something trivial like fizzbuzz or wordcount on the whiteboard. The candidate then became extremely defensive and tried to argue that such task was "beneath him", arguing for a good 15 minutes why he shouldn't have to do it.
Then the dev just left the room and said that he used this question as a warmup with new hires and it typically takes them less than 10 minutes.
Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
Now the hard part isn't so much to create an interview process that works well, but to create a pipeline that feeds into this interview process that has a high signal to noise ratio. In my experience, the best predictors of a good signal to noise ratio was to select for CS fundamentals, good references and offer above market comp. The latter is especially crucial now since there's no more "local market" to speak of now that remote work is a lot prevalent. The "local market's" best devs are working for SV firms at SV salaries mentoring SV employees.
[0] https://blog.codinghorror.com/why-cant-programmers-program/
[1] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
- Steve Jobs
I think the real disconnect with the 'inclusive culture' boom comes because humans are involved so heavily in the process. The _idea_ is great, we want to be fully aware of our internal biases and avoid having them color our perception as much as possible so we do not shoot ourselves in the foot.
In practice, I have yet to see inclusivity programs at corporations be anything more than virtue signaling, and an opportunity to exclude others under the guise of "inclusivity" wink wink.
Remember 'affirmative action'? It's palpably Orwellian that inclusivity is newspeak; what we call it now.