It could be adding a feature to a side project, starting something new, or contributing to open source. (Or, we have some stock ideas if people need inspiration.) Rather than asking them to solve a problem they had never heard of before or use a codebase they're not familiar with, I get so much more out of watching them work in their own environment.
I can ask them questions about why they're doing something, and they tend to have much more detailed answers because they've been thinking about it for weeks. They're solving a problem they care about in a codebase they know, which mimics how working with them will be a few months in.
We can talk about tradeoffs and design decision, and I get a real sense for how they think. Plus, I've found most people are so much more comfortable and excited.
(If you're interested, I wrote a blog post about all the ways I've designed how we interview, to give everyone the chance to show off the best version of themselves: https://blog.readme.com/designing-a-candidate-focused-interv... And of course, I'm hiring!)
For people like this, though, I’ve seen many use it as an opportunity to work on something they’ve been wanting to for a while and never had the time! A Hello World in a new language, merging in PRs on an old repo, writing a script to automate something or even a take home coding exam for another company.
I have a gigantic open-source portfolio[0]. I also go to great lengths to make sure that I’m a “known quantity” (like posting frequent exposition on fora like this one). I don’t feel that I have anything to hide. Everyone that has worked with me has found the experience enormously fulfilling.
It’s been quite jarring to see this ignored and treated with disbelief. That’s pretty much exactly the opposite reaction from the one that I would have, but I guess I’m out of touch with the way things are done, these days. I even talked to someone who told me about a friend that had invented some kind of machine learning candidate evaluation system, and relied on that, as opposed to human vetting.
One of my goals is to discourage people that wouldn’t want to work with me. I was a hiring manager for a fairly high-functioning, high-stakes team, and am quite aware of the value of making good hiring choices, which go well beyond simple expertise. Team dynamics are also very important.
I feel that getting a job as a result of misrepresentation is a big mistake, and a recipe for misery. So I don't hide my age, don’t pad my résumé, and make every effort to be as open as possible. If you don't want to work with me; fine. Let's not waste each others' time.
I have always sought to make it easy for people to figure me out. Since I live a fairly good life, and know my way around the playground, I feel that I'm a pretty good bet, but I have no problems if others think otherwise.
One of my favorite Twain quotes is "Let us so live that when we come to die, even the undertaker will be sorry."
I think I just threw up in my mouth a little.
My guess is most people are covered by some sort of IP clause for their main work, and the most visible OSS projects are still not run by very many people, so most devs are not gonna be able to show you much.
Rather, the engineers on the team are usually building something new, and figuring out how to accomplish it is their responsibility. This way of interviewing mimics that a lot closer.
If someone can't explain to me a project they're working on, then they probably won't be able to do the same while we're working together. And that's just as important of a skill as writing code.
If I'm expecting interviewees (who are nervous!) to learn and understand my codebase/problem, why can't I do the same for theirs?
do you (or others on team) ever make suggestions on how the code should be written? I think this would be an interesting way to pre-teach the hire on how you like things structured.
on edit - I guess since you ask them questions that implies making suggestions as well, but just wondering how it works.
Imagine you’re taking a class on a topic you don’t know. You can tell pretty quickly from how the instructor talks and explains something if they know what they’re talking about. It’s very similar! I’ve certainly interviewed people much more skilled than me in languages I’ve never used, and still got a pretty great sense of their expertise.
It’s an advantage we have as a small company! We can rate people based on whatever criteria gives them the best chance, and we can find a ton of great candidates that maybe wouldn’t make it through a traditional hiring interview at a larger company.
If you asked me to work on something, it would be a toy project. Because I can't contribute to open source and if I build anything substantial it will be taken from me.
You shouldn't punish people in this situation. It's extremely common, maybe even the majority of coders outside California.
That being said, I've interviewed hundreds of candidates over the past 5 years, and not a single one has cited IP assignment as a hesitation.
of course they do, you're a knowledge worker for them. they need to own what you create.
a standard phrasing includes "related to the employer's business". see for yourself: https://www.lawinsider.com/clause/intellectual-property-assi...
"during the period of his employment relating to the business of the Company". If your contract doesn't have the words "relating to the business of the Company" then add those words in yourself, signing and dating the change and telling the company of your change.
you're not their slave you know. they don't get stuff you make in your free time. however, not replicating the company's business is a fair requirement, since they might share enough information with you to do just that. I think it's fair for them to get everything related to their business.
It's a great proxy for empathy and understanding. If you're curious, it means you want to learn the why behind why other people do things.
If we sense they're doing something they've rehearsed a bit too much, we might push them to do it a bit differently. "What if you tried to do this by _____", just to make sure they can think on their feet.
Honestly, though, the best candidates for us are the ones who wouldn't try to game it. They hear the prompt, and are just so excited to show and talk about a project they're building that it's not a problem!
By letting the interviewee pick, we're letting them decide which of their "superpowers" to show off. If they can explain why they chose to do something the way they're doing it, and show they understand the tradeoffs, that's good enough for me.
My "FizzBuzz" coding question that I ask on phone screens is "Write a function that takes a string and returns a list containing the most frequent character." It has a few clarifications that are needed and I expect the person to ask about (What if there is no most frequent character like in an empty string (return an empty list), multiple characters equally frequent (each equally frequent character should be in the returned list once), etc) and has some basic logic requirements.
I am continuously astounded when people can't do this. I encourage them to use the language they're most comfortable with. I give them time to think about the problem, encourage them to ask questions etc. If they seem stuck I'll ask them to walk me through what they're thinking about, etc.
I've had multiple people with master's degrees in CS or with years of professional programming experience get stumped by this problem and it just makes zero sense to me. I took undergraduate CS courses and this would've been an easy problem in those classes. I was a TA in an undergraduate intro to programming course and I would've expected all of my students to be able to solve this problem in similar circumstances.
It seems to me like someone with a Master's degree in math has come in and I'm asking them to "solve for x" in the kind of equation you might give to an eighth grader and they're unable to do it.
Unfortunately people tire me very quickly, I have a limited budget of people time per-day and interviews frequently blow that budget early in the day yet continue into the afternoon (talking particulary onsites for bay area companies here).
I would say 9/10 I manage fine but there have been occasions especially when I have been in between jobs and taking lots of interviews when I have just bombed because a particular company stacked all the code heavy interviews late into the panel.
Separately from that speaking as a hiring manager for several roles I don't think coding on demand is a good metric. I don't expect people to do it once hired so it's not useful as a measurement of their fitness for the role either.
Design heavy interview questions on the other hand I find very useful. They generally are less specific and speak more to experience, knowledge and taste. Things that are very relevant in an engineers real-world productivity. They are very punishing vs code on-demand interviews for new grads though so if your pipeline expects to take a lot of new grads you may need a different approach.
However in my experience questions like "How would you architect a replacement of Amazons S3 service with similar durability" or "Given a smart grid reporting metrics from every home in the US how would you store this data for analysing both long-term trends and intra-day anomalies" yield by far the best signal to noise ratio.
This isn't obvious at all - I've interviewed many people that have faked their GitHubs or were just completely incompetent at software development despite their amazing GH, OSS profiles or CVs.
People lie, a lot. I'm sorry, but having "a profile" isn't a useful signal for an interviewer :(
> However in my experience questions like "How would you architect a replacement of Amazons S3 service with similar durability" or "Given a smart grid reporting metrics from every home in the US how would you store this data for analysing both long-term trends and intra-day anomalies" yield by far the best signal to noise ratio.
Agreed, in my experience these questions give by far the best signal for a good coworker in general. Sadly they're not appropriate to interview junior developers.
So yeah, it might not be useful for recruiters/sources that don't understand the technical side deep enough to evaluate but it 100% sure be good signal for an interviewer if they know the fields and projects in question you are hiring for (they probably should...).
(I was also a TA in a course that did these sorts of trivial challenges, and going through that experience really made me take a hard look at how I perceived those 1st year students)
But the thing about interviews is that you always want to heavily bias towards false negatives. A single bad hire is just astronomically costly. I think it’s fair to say that many smaller companies have literally been bankrupted by one incompetent person. When it comes to engineering positions, this is especially true.
I’d rather nuke 20 good candidates having a bad day, then risk letting one bad engineer through the door. That’s why I’m skeptical that the interview process can even be “fixed”.
Fundamentally it’s a mis-alignment of incentives. Employers have very strong to rule out candidates at even the slightest hint of incompetence. But for the interviewee, being judged so quickly, harshly and unfairly on a such a personal level makes them feel vulnerable and hurt.
There's only one (not counting ties), so you want a single-item list? Or if 's' is the most frequent, because there are 5 of them, do you want [s, s, s, s, s]? Or since the list just needs to contain the most frequent character, seems like [r, s, t] would be correct as well. Now that I think about it, just returning the original string would have to contain the most frequent character. Also, do capital and lowercase of the same letter count as different characters?
Just seems like a rather unclear directive.
I think of it as the FizzBuzz version of getting requirements for the thing you're supposed to be doing.
I would expect a skilled programmer to be be able to make intelligent inferences about the intended behavior in the face of ambiguous problem statements. FWIW, I would probably explain my reasoning as I went -- would you interrupt a candidate to say you'd rather see a different behavior (ex: my prototype would return a list with an empty string, but maybe you'd rather an empty list), or just silently judge them? Also, it look me about 2 minutes to produce a reasonable prototype -- maybe if the challenge were harder I'd expect someone to ask more questions up front, but in this case it's trivial to iterate.
If the only thing you are testing for is the instinct to clarify unclear requirements, you'll surprise people who are used to the leetcode-style, thereby getting false negatives. It's just as much a trick question then, just in a different dimension.
Expect professionalism from both parts.
First, I would feel like if I have to ask questions if you then I’m missing something obvious you want me to be able to deduce. And if I take too long I’d fail your test.
So I’d jump in merrily and work up a quick and dirty version before even realizing that the output has ambiguities. Hopefully at that point I’d ask how you want them resolved, but maybe I decide you want to see me to make my own decisions and I’d come up with my own way to handle them.
I had just received offers from two FAANGs, and I was flown into the Bay Area to interview at a third. One of the interviews was mostly behavioral, with 20 mins at the end for an algorithms problem that I had done before on LeetCode. I did so badly that after 20 minutes the only thing on the whiteboard was the problem statement. I still remember the interviewer taking a picture of it, and me kind of cringing because of how bad it looked. I didn't get an offer, unsurprisingly.
Because they disqualified themselves from hiring me. What I remember from that phone screen was getting mentally _TIRED_, sighing, and then lying about not being able to do it.
Seriously, don't fucking ask me to describe code or algorithms over the phone. The number of times I've had someone do some variation of this over my lifetime is such that I don't want to have you as a co-worker or a manager.
Do you know why I have this attitude? Because the number of times I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed, is enough that I've decided if you think this is the right way to go about hiring, you're doing it wrong and I don't want to be in that hellhole.
I imagine if I was hiring someone to play in a band or sing in a choir it would probably be a good idea to ask them to play or sing a bit before extending an offer. If I was hiring someone to work at plumbing company, I'd probably ask them a "How would you fix..." type question or two.
Saying that these questions are forbidden just because you are sometimes tired of doing them strikes me as unreasonable. While, of course, you get to decide what signals would make a company a bad fit for you - e.g. if I asked you my question you might think my company would be a bad fit for you, the interviewer decides which answers are a bad fit for the company. I would definitely take an answer like this as indicating a bad fit for the company.
This line from your comment is interesting to me though: "I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed". I frequently ask questions where I expect the candidate to want to clarifications or challenge what I'm saying. That's part of my job, and I'm trying to test for it the best way I know how. If you're happy with your job interviewing performance, then more power to you, but if you aren't - you may consider changing how you think about instances where the interviewer is wrong. Perhaps they are presenting opportunities for you to engage with them in a productive way, and even if they aren't doing so intentionally, maybe they are doing so unintentionally.
You know what's unreasonable? Interviewing a bricklayer and asking them to describe how they lay bricks over the phone. And this is why your comparison to singing is hugely flawed. The phone is literally designed for carrying and hearing voices, not for more complex things.
You act as if it's literally impossible to figure out if someone can code unless you're trying to do it over the phone.
> This line from your comment is interesting to me though: "I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed". I frequently ask questions where I expect the candidate to want to clarifications or challenge what I'm saying.
I once had someone ask me what _type_ was returned from a controller in asp.net MVC, and when I gave the technically correct response this person told me I was wrong, a view is returned, then immediately ended the call. The number of times I've seen stuff like this happen is way too high.
But more than that, I want to address the attitude in that last point. This is a _HUGE_ problem with interviewers. They often come to conclusions they ought not come to based upon the circumstances. And the defenseless interviewee's are left trying to suss out hidden requirements like "expects them to ask clarifying questions". Interviewee's come in and have to try and navigate these byzantine, hidden requirements. And the question is, why do interviewers have hidden requirements like this? Because they know if they're _HONEST_ and communicate what they're looking for, the candidate will give it to them. That by itself should be a wake up call that interview's are hostile and un-real.
I'm just in a position in my career that I have the leverage to refuse to play those games. If you do something like that, you fundamentally have no clue how to actually hire people, and I refuse to bring any value to you or your company.
It allows your company to learn, to get a feel for the person's ability to communicate, and to stay away from stupid things like tests.
It's also easily the best interviewing process I've ever experienced.
---
edit: Since it appears that multiple people have taken this to represent an hour+ long presentation, let me clarify here.
A better way to think about this would be a long-form discussion about a topic. The initial "talk" would be 5-10 minutes, with a Q&A afterwards to ask for clarifying questions.
I've been working in this field for over 25 years. I spent roughly 30 minutes of prep for the interview and it was easily the best interview process I've ever experienced.
If you can’t boil your work down into accessible presentations and adapt them for technical or non-technical audiences, you won’t get anywhere.
Nobody will agree to help you, allow your team to grow, give you budget for tools you need, make compromises with you around integration requirements or shared design patterns, unless you can frequently convert your software work into highly professional presentations.
I’d argue that competent presenting and writing skills are of equal, or even greater, importance than software skills or expert knowledge, whether for a research software job, software contractor job, SWE within tech / ecommerce / banking etc industries - across the board there are pretty much zero types of software engineering jobs in which software skills are actually the most important thing. Presenting and writing are frequently much more important.
This isn't true in my experience, there are plenty of well paid jobs where you don't have to talk to non-technical people. For example, lets say you are developing infrastructure at Google, how often do you think you talk to non-engineers? Never, everyone in your management chain will be engineers, your product managers are ex-engineers, your users are engineers etc. This is true for directors managing lots of people as well as well.
And even when you develop services much of it is just getting the technical parts right so you can work there as well at Google with very little time spent talking to non-technical people, so senior positions there are mostly about technical stuff.
You could argue that Google and other big companies are a special case, however if we exclude low paid jobs where you just earn half of what you'd do at Google or Facebook then pure technical work like this is a very significant fraction of the market. I'm not sure why I'd work on my presentations skills so I could get hired by a company paying me less.
To be clear, explaining your work is a part of every developers job. But top notch presentation skills don't need to be a part of it. Developers can talk to other developers in sub-optimal ways that are still effective. But when your hiring process has an hour long presentation as a part of it, you are optimizing for the thing that is very likely not the most important feature of the role. The point is to structure your hiring process such that it optimizes for the right things for the role.
Technically speaking, if we're in an interview to "optimize for the right thing", then we would just have them come in and type.
But we all know that's also optimizing for "the wrong thing" despite it being what we do physically day in and day out.
This is because you _CAN'T_ "optimize for the right thing" in an interview. It's not possible and this needs to be acknowledged.
Therefore you go for something else. Anyone who is a software developer can listen to a 5-10 minute presentation from another developer and start forming general, broad opinions. They can then start asking questions afterwards to help clarify those opinions.
But the Q&A also has another purpose. Way too often interviewers will come to conclusions that don't follow (non-sequitur). I once had someone tell me they didn't feel I would be a team player because I told them as long as I had headphones, open office would be ok. The main difference I've found between business people and technical people is that business people will ask clarifying questions. The Q&A sets the expectation for the _INTERVIEWER_ to ask clarifying questions.
Some developers are more comfortable with communicating via 1:1 conversations, design documents, proof of concept PRs, etc.
I've done a lot of public speaking and presenting, but am aware I am in the minority of developers. I've also managed a lot of people and many of them have struggled with self confidence and presenting to large groups. This has not been a reflection on their performance as an engineer.
I've done a lot of both traditional interviews (presentation on research or previous work followed by 1:1) and "tech" interviews (coding challenges and whiteboarding) from both sides. Tech-style interviews evaluate a much narrower swath of skills are are much more easily gamed.
I feel that a presentation evaluates broader things: how well do you communicate and how comfortable are you discussing and answering questions about a topic you know well.
It's a better indicator of job performance, i.m.o.
Our group hires people in the software+hardware+mechatronics space, and we look for people that bring diverse skills we don’t yet cover well. The kind of person we want can teach us something new in 30 minutes, and a good project for that could be backyard hackery or university research. Either is good. I totally love the format. (Then again, I am a “Talk is cheap. Show me a working (even if janky) robot.” kind of person.)
Related, a former BioMed engineer coworker said Medtronic used a similar process - start the interview day with a presentation.
Technical talks are never about coding, but more a conversation about previous project and perhaps a little side track on how to tackle large projects.
The style of interviews discused here on HN show an alarmingly low level of trust in the people you want to hire. That’s a bad way to start a relationship.
But there's also a bunch of other dynamics going on. 'Geek machismo' is a very non-trivial one. Other's are akin to hazing. One friend of mine described one value of very high bar hiring is that people in the bubble can (go back to) assume that the colleague is smart, etc. rather than assuming people are stupid/incompetent until proven otherwise--and that can be a good thing from the culture/sociology standpoint.
I suspect there are technical tests in developer interviews because there can be rather than because they're actually a useful way to appraise someone.
It also seems weird to lie in your CV, what are you going to do if you actually get the job? You’d fail horribly and get fired.
My point is that in some parts of the world you can actually trust resumes. If companies start making outrages demands, like 15 years of experience in a 10 year old technology, then of cause desperate people will lie.
I have seen small programming task presented to candidates at a previous employeer, after I was hired, but that sort of failed. All the candidate actually submitted perfectly good solution and the process just reverted to just talking to each candidate in a relaxing setting.
Edit: I actually once interviewed for a job that required a personality test, because HR need to justify their existance I assume. I will never apply at a company which uses personality tests again. Not only was it wrong, it’s also kinda traumatic. But this was after actually getting the job, the test was sort of a “before we actually sign, please take this test”
So my starting salary was close to what typical software engineers earn as a seasoned senior, and then I got to twice that within a few years. That is why I care about technical interviews.
Provide candidates a small sample "project" of 50-100 lines, a Flask webapp perhaps. Work with them to get it running on whatever interview machine they're using. Have them extend it with some fairly trivial feature, around FizzBuzz complexity but with a bit of a design component. These questions should be carefully designed and standardized across interviews.
Key point: completely strip out identifying information from their solution and pass it to someone else for grading. The person "giving" the question is a proctor, not a judge, and they're officially on the candidate's side. Possibly add a side dish of talky conceptual, design, and behavioral questions, with as much blinding in the assessment of the answers as possible, though that's harder if it's a dialogue... Hmm.
A good interview is fun. You get a nice bite-sized problem to solve with someone to bounce ideas off of, and maybe learn something in the process. I'd suggest almost a game level-design approach to coding problems. Make interviewing fun again.
It had a few benefits:
First, this is much closer to the type of work that we'd ask a new employee to do, as compared to the typical industry interview question: "did you pay attention in computer science? You did? Now prove it by solving this problem that's a thinly-veiled algorithm exercise. Using a computer? Oh heck no, you're going to use a marker! There's no trick to it, we don't ask interview questions that have tricks, but if it leaks externally, it's 100% useless and we cannot ever ask it again"
On that topic, who cares if the premise leaks? If the code somehow got out, we'd just write another one.
We get to hear them talk, out loud, about their expectations and how the app's organization differs from them. This isn't interesting by itself, but the candidate gets a chance to articulate their experiences and how well they notice patterns. For more senior engineers, you get to see how quickly they can pattern match and explain the structure of the client and server, and explain how it could evolve under different circumstances. "You're in a startup that just got a ton of VC based on this hackathon project. Your friends hire you and give you this codebase. How do you spend your next 2 months?"
Also, you'd be surprised how many people disqualify themselves on attitude. People will read sections of the toy app and say out loud, "who the hell wrote this? This is terrible." And you'll ask them, "What do you mean by that?" and they'll rant about some section of it, and you're like, "wow, I really don't want to work with this person." If these engineers can't even keep it together during an EXERCISE whose PREMISE is that the code doesn't work well and can be improved, it's a negative signal that they're going to demonstrate any empathy when they're working with you or any of your colleagues over the years.
As an interviewee, get a strict proctor that doesn't want to give out information? Well you're not going to appear as competent as the interviewee who got the proctor that wanted the interview to be fun
I would say it's still an improvement over leaving the whole thing up to the whims it the interviewer.
My favorite interview question I ever had was to design an elevator system. There's no right answer. It wasn't even mostly code, it was mostly talking. But every so often, they'd say "write that as a method" and let me imagine whatever api I wanted. I had so much fun doing that.
It would be an interesting idea just to have a candidate go to the site and screenshare to watch them do some coding.
All excuses for why this won't work can be answered with: then make a better test.
As a species, across all cultures, we tell ourselves that the human we're after -- the student, the employee, the promotee -- can't possibly be selected by test alone.
When in fact, if we can't write a test to select the qualified humans, then either we're too lazy to write one, or, more likely, we actually want to leave plenty of room for human bias to do the actual selecting.
And this is ok because we have a special power: we can judge the value of every human, and its future likelihood for success, with a single conversation. If we weren't in a tech company, we could make a very good living reading palms.
We never hire people who don't pass our wonderfully fuzzy exams, so we have no evidence that we're selecting the best people or not. No worries, though, our palm reading is very, very accurate.
The way we look at it is like this. We make a test no one can pass. We always have one more question or one more "level" that there's not enough time for. Because when all candidates fail, we have to fall back on our palm reading, which is just how we like it.
Power and privilege are precious resources to us. We give them out to those most likely to reciprocate. That's what we're poking for with our "culture fit".
In the future, students of our culture will look back on our hiring practices and say, "It was illegal to hire based on race, age, sex, and a million other things, but not beauty??? They didn't start with beauty? And they never realized that beauty needed to be in the mix? I don't understand."
But we understand. It makes perfect sense.
The same reasons whiteboard interviews are problematic will apply to any test done at scale.
if we can't write a test to select the qualified humans, then either we're too lazy to write one, or, more likely, we actually want to leave plenty of room for human bias to do the actual selecting.
Creating such an exam is an unsolved problem.
Why can't we write a test to see if she can code in Python at an appropriate level of expertise without mixing it in with algorithm gotchas? What are the algorithms she needs to know? Why can't we provide her with a list of those algorithms and show her how we will test for mastery of them?
>Why can't we provide her with a list of those algorithms and show her how we will test for mastery of them?
That's exactly what is done, it's called leetcode.com, recruiters send you a link to it and tell you to study. I don't see how anything you're describing isn't covered by the existing whiteboard interview dynamic.
We provide a candidate a piece of production code with some slight tweaks made that will break it in (common) edge cases. Further, we give a test harness that already supplies some basic tests for which it will work. What we're looking for is to see if the candidate can grok the provided code (< 30 lines for a 2hr interview) and reason through ways it could break. Writing the tests insures that the candidate can _actually_ write the code and shows any areas they might be less focused on or miss.
Compared to our old procedure of writing the (provided) code snippet this has been a far more successful approach. In 2 hours I learn more about an applicant than through the entirety of the previous process. An added benefit: this workflow is representative of real work - you'll need to track down code you likely didn't write and implement new functionality, or fix something that's broken. Doing this in an interview is a near direct translation to the job (minus understanding the whole codebase).
I'd strongly recommend this unit testing based process and it can definitely be tweaked to fit your own scenario!
This. The stereotypical algo interview assumes a greenfield project, but there's almost always "legacy code" you'll need to grok first.
“Indeed, the ratio of time spent reading versus writing is well over 10 to 1.” ― Robert C. Martin
It's just bizarre to me.
Part of the deal is the 'humiliation' of being stuck following someone else's stylistic/design/naming-scheme choices. Another aspect is the curse of knowledge, leading to useless documentation. Plus, it's easy to underestimate the amount of effort required to reinvent the wheel: all the necessary complexity looks like incidental complexity at first glance. "Why should I be forced to learn all this gobblety-guck when I should just be able to re-implement from scratch in an afternoon? [... three weeks later] Oh, now I see why."
Do you know what I do when I see a problem for the first time that I don't immediately know the answer to? I Google it. Because A) it's faster B) many minds greater than mine might have already come up with an optimal solution that I'm unlikely to self-produce in 2 hours.
What if a developer hasn't been deep in debugging production code for a while? Those edge cases might not be top of mind.
Maybe they're doing documentation or planning or writing Jira tickets and remembering all the ways to nullcheck a JavaScript variable isn't their current life goal. Instead they might try running the code locally or write some tests for it or research to see what others have done in similar patterns.
Dreaming up magical solutions while under the stressful glare of people judging your life's work may sound reasonable to you because presumably it matches up to some internal signals you value in each other, but every workplace is different -- different codebases, different problems, edge cases prevalent in one may not be prevalent in another.
You're better off doing a traditional Q&A about their experience and a take home mini-project (or asking them about theirs). If they don't want to spend time on that, then and only them give them some timed test you feel is applicable.
Of course there's Q&A and their reasoning/explanation is just as valuable as the code they right. This is just the technical portion of the interview. However, I think we've really hit the sweet spot with this approach.
Take logging. Super important - I spend a lot of time evangelizing logging to younger devs and how to do it well. But filtering? How? In what way? For what values? I don't have context but the expectation I should be able to filter for specific things you have secret preferences for strikes me as a signal that's not clear or universal.
Checking HTTP codes. In what context? There are plenty of times when it's not helpful -- for example there are many page not founds that DON'T return as 404 (even tho they should). So this a priori belief that all devs should automatically have this built-in preference for checking HTTP codes as some universal signal of quality in programming is, I think, a larger assumption than you might realize.
I don't think there is a perfect way to interview, but I do think if you're going to quiz developers it helps to give them some advance warning of what areas you prefer to focus on so it's not such a random, out-of-left field line of questioning.
You may not consider your questions abnormal -- but that's the problem, neither do the people who ask obscure algorithm questions! It's very hard to validate your assumptions.
You also don't give a second chance if they fail this one assignment.
Writing some algorithm No why should I? If I need an algorithm that must be fast I will look up a peace of code that is used and tested by other people.
No I will not live google in front of you watching me. How is this a real world scenario? How came up with this?
Like stupid questions like: How many log entries should a service have each hour? Yes this was a real question in a job interview.
IT Job interviews are broken in so many ways.
I refuse to live code or have this you now have X minutes to solve a problem stuff. Again this is not how I work and also I have never seen people work like that in the real world. Only in shitty teams with shitty organization and where they were thinking that they were agile because the sprint ends in 5 minutes...
Here is the only way of interview I tolerate these days: 1) (optional) Have a call with HR
2) Talk with people from the team in an open way. Talk about real challenges and how the applicant can help in the challenges of the team. Check if the applicant will fit your team! Good team chemistry eats technical knowledge by 1000x. If you only take people because they know some algorithm you will most likely end up with something in Germany they call "fachidioten". They will solve the problem on a technical level but will bring with them 1000 other social, product and long term problems with them.
3) If you still not convinced that this person is good enough for your team then. Give that person a small challenge he/she can do at at any time she/he wants.
4) Let the applicant send you the solution and if you think it is okay then talk about the solution of the applicant and see how he/she reacts to criticism. Again a team that has good social skills and likes each other will eat every team were you have rock stars that think they are the best.
I agree that the in person time pressured performative problem solving & coding engineering interview process is often a poor approximation of the work.
If at least two people who worked closely with someone like them enough to give a referral, they are probably worth hiring.
I had quite a few “aha” moments where I was “ah wait, I totally didn’t do that, have I?”. Didn’t get the job, however neither did I like them.
We had one case were you could clearly see that the person had no clue what he is talking about it.
Its quit easy to see.
Just ask them to change something small.
45 minutes - do code review of this extremely buggy code like you'd review a coworker's code
45 minutes - presentation on a technical topic to one or more team members
45 minutes - behavioral. Let's talk about failures, conflicts in your career. Who are your role models? What do they do well? Leadership abilities, career progression
45 minutes - troubleshooting. Here's a docker container. Make the service inside it work
Am interested to learn more. Are there open positions?
There are many awesome people you can hire that you wouldn't hang out after work, and that's OK - you don't have to be friends with all your coworkers, you just have to be able to work together.
Instead, identify qualities & behaviors that your company values or are exhibited by successful employees and ask for examples of the candidate demonstrating them.
Instead, I think we should attempt to really dig down into elements of "culture" and identify the ones that are relevant to the job vs those that are not. For example, some relevant elements might be:
* Preferred development process.
* Pace of delivery and willingness to ship bugs.
* How disputes are settled.
* Using libraries vs build it in house.
* Process for adopting new technology.
* Preferred communication methods. Email vs chat vs video calls vs in-person (back in the pre-plague times).
And there are plenty of things that are _not_ important:
* Hobbies and how people spend their time outside of work.
* Favorite films, music, books, food, drinks, etc.
* Whether you work on side and/or FOSS projects (unless there's a legal issue with side projects).
I'd also note that for some of these things I expect the candidate to ask _me_ about them if that's something they care about. In fact, those sorts of questions are a very positive signal to me. It shows that the candidate has some insight into their own work and what makes them happy and productive. I'm very happy to find out that this isn't a good fit _before_ they've started working with me!
> * How disputes are settled.
Suppose you pose to the candidate a situation in which they are in the right, and ask them how they would proceed. They respond by giving the impression that they wouldn't easily compromise on their correct point of view. Is this positive or negative?
For many interviewers the answer will unconsciously depend on the candidate's gender. If the candidate is a man it's positive because they are "assertive" and "stick to their guns", whereas if they're a woman it's negative because they're "pushy" and "unwilling to listen to other points of view".
Corporate culture is not about being similar to your coworkers, it is exactly what you said - company values and ways of working together. Don't get tripped up by the word "culture" - it has multiple meanings.
Yes people in teams should like each other because otherwise you will have a bunch of unmotivated pixel pushers as developers that only think when the time runs out.
In all of the companies were soft skills did not matter and I worked for people were coming and going.
No you don't have to be friends outside of work but usually you are still spending 8 hours 5 days a week with these people and maybe for some people it is okay to not like the other people in the company but from my experience this leads to no good in the long run.
When hiring what works for me is looking at past experience. If someone says they are a cpp performance guy, that creates an expectation of what they should be able to chat about. If you're making it up, you will run out of stuff to say, and I'm patient enough to let people keep talking. Hopefully I'll learn something too, it happens often with top pros. But basically if you're experienced in a field it should be hard to come up with anything where you draw a total blank. If that's the case you're pretty good in my book, the main differentiator after that is whether you've done exactly what the firm needs or just something similar.
With entry level people it's hard. That's why they typically make less money, it's an Akerlof lemon cut due to the being a lot of people who just aren't going to be good at it. This isn't even necessarily a coding thing, but general life maturity. We had a guy who couldn't wake up on time to get to work, and he couldn't phone in when he was late either. I reckon there's no way to know about this kind of attitude problem until you hire them. This is also why you can expect a relatively high jump in pay after your first few years, it's proven that you aren't one of the lemons. The distribution is a large mass of hard to differentiate professionals, plus some duds of near zero value.
So for entry level, your best bet is simply finding someone you think is smart. You'll end up falling back to the same thing as everyone else who is hiring: the prestige of the uni they went to, and some pot luck questions to make sure they picked up something. However that's really not enough to know what you want to know, which is whether they are able and willing to learn what you need.
Of course, it doesn't mean the opposite holds. They could be interested in something but not end up being interested in the job.
If there's one thing I don't want to do, it's work by assignment. I'm not a school kid, I'm a professional software developer. I'll tell you what needs to be done. You can inform me of the goals you have for the company, and I'll make sure that happens.
What I'm saying is that despite all my successes, you would consider me a 'dud'.
That means that regardless of the format, interviews are always going to be inherently unfair. Even the slightest hint of incompetence will nuke your chances. Assessing a person you’ve never met is just an inherently noisy process, and there’s always going to be false negatives. And almost every company in the world would much rather be safe than fair. You probably are having a bad day or misunderstood the question, but employers can’t afford to take that risk.
For the interviewees part, the process is just plain soul draining. You’re literally subjecting your life’s work to the harsh, quick and unfair judgement of people you’ve never met. You’re putting yourself in a position of extreme vulnerability, and it’s easy to feel like a piece of meat.
I don’t really see how twiddling with the format will change the fundamental dynamics. The only thing I can say is that both sides should have more understanding and sympathy for the other.
Candidates, don’t take any single rejection too personally. Remember the interviewer is a person, who also can be having a shitty day or make mistakes. Interviewers be kind and positive and give constructive feedback, even for the clear no’s.
If you really hate interviewing, then spend more time networking. Hiring personal contacts and recommendations is much less noisy/risky, and therefore an overall less adversarial process.
When I searched for anew job I had some slow time at my existing place and had all the time in the world to work on a take home assignment, while I could see the the candidate I recently interviewed had a hard time expanding their assignment beyond the basics due to lack of time.
Another variant of this challenge is; when you have multiple interviewers and some are positive some negative about the candidate. Generally companies will always side toward rejection unless there is a specific reason not to (for example, you need someone super technical but they struggled on the soft-skills side and you have the right team to help them grow in).
Surely, it depends on the situation of the specific employer. So the trick for a candidate would be to figure out if there exists a single argument (e.g. a certain skill) which makes it impossible to not hire you.
Coming back to the original question: You can be honest to the candidate; figure out what your organization is desperate for and tell them. Of course, this gives up some negotiation strength.
That works best for programmers with entry to mid-level. In Senior level you need to focus more on problem solving.
Lots of people say this however I've never seen anyone come up with evidence that their team actually is excellent.
You want someone that can easily talk about the pros & cons of using different data structures or algorithms to solve problems. Who can move up & down layers of abstraction without getting confused & lost. Who can hide complexity to make clean, readable, maintainable code.
Many people, especially junior/less experienced engineers, have only memorized solutions to problems. They don't have a strong toolbox of solutions and the experience to know how to apply them (and evaluate the options).
A good problem solver is methodical - they don't just guess at random data structures, they identify what the requirements are of the problem and then choose the appropriate data structure (or compose one) which meets those requirements. They'll have a process, which may be different than your process, that they use to break down problems into smaller, solvable pieces.
Knowledge is usually the easiest problem to fix. It’s a google search away. All the bad hires I’ve seen have had masters or phds and the problem has been poor work ethic, focus, and practical application of knowledge towards solving business problems. Interviews, in my opinion, can easily test for basic dev knowledge but should focus more on the way the candidate will conduct themself as an employee.
For senior engineers I would say it's all about design/architectural problems and talking generally about concepts and taste to gauge fit.
Mid-level engineers I think the take-home is king. A scoped problem, 1-2 hours at most that can be followed up with an interview talking about an extending their solution is pretty bulletproof. You get to understand what tools that are comfortable with, practices around source-control, testing etc that you would expect from an experienced but not overly senior engineer.
Finally for entry level candidates I don't really know. Personally I hire entry level people on gut feel. Algorithms and datastructures just tell me if they a) went to university and b) paid attention. Seeing as I did neither of those things and still write IMO decent software that seems like a hypocritical measure. So I stick with how I feel about them, their level of enthusiasm, how well they researched the position, what stuff they have played with etc.
At the end of it the biggest burning question they had was "where are the unit tests". I was like ... you see this circle here with this label inside it? This was critical to the entire system and therefore had unit tests surrounding it.
didn't get the job, they seemed kind of annoyed that I didn't get into great detail about things like unit tests and instead talked mostly about the architectural problems I solved.
I wish companies actually valued that level of knowledge.
Hire them, give them a week to catch up, and their off; or wait months to find a perfect match? Any developer with a few years experience knows how to pick up the next piece of technology, the same way everybody else does: read docs, search blogs and forums, ask StackOverflow, etc.
Now if all you find is Perl guy, that is another problem.
1a. Optional coding test at home (no time limit)
1b. Optional coding test in front of me (1hr time limit)
2. Very simple fizzbuzz-level problem. Essentially can you break down a problem with an if statement and while loop. This has the least bearing on the process.
3. System architecture/design "whiteboard". We ask a question that's essentially an entirely different problem space from out company but who's end system design, and lifecycle, looks almost exactly as the design our company took. Essentially if you `s/A/B/g` you'll have a 1:1 idea of what our company's infrastructure looks like by the end of the problem. There are absolutely no wrong answers, just answer how you would if you were given this ticket ("design X") and I'll ask from time to time "how many QPS can this handle", "any bottlenecks you can identify", "how much bandwidth/cpu/memory will this take", etc to see if they understand the implications of their decisions.
Surprising facts:
1. Most people can do the take home coding test "enough" but do horribly on the on the phone coding test. This has been shocking to me because I'm not someone who struggles with this aspect of the interview process (I'm horrible at explaining myself).
2. When we get into the system arch/design session 99% of people have no idea how to proceed. They've never sketched out the design for something in front of someone else and have often never thought about latency, memory, CPU, disk, etc. We had one engineer take it who couldn't even come up with a way to estimate how many QPS a service might receive in the worst case. Another engineer settled on a design that was going to cost >$1mill/month in AWS DB costs and could not imagine any way to reduce this bill and thought this was reasonable (just needed to put a cache node at an edge location).
3. People use buzz words. They use them a lot. If I had a working feature for every time I've heard someone just say "autoscale it" whenever they hit a bottleneck during the arch/design section I wouldn't need to hire an engineers. It shows a core lack of understanding of where the bottlenecks in our interview question are coming from.
4. We see some (small) percentage give up on doing the coding test. I'd say it's less than 10% of inbound leads. Obviously a significant portion of people don't want to code in front of someone and don't want to code at home. Unfortunately I can't advocate for hiring someone until I see code that they've written. We've had horrible experiences hiring "senior" engineers who actually don't know how to program.
5. I've had people freak out, and start cursing at me, when we get to an interview question they get stuck on. It's something I'd never have imagined happening in a professional setting but, at least, they've been filtered and won't be working with me in the future!
6. Resumes and pedigree have the worst correlation to quality of the engineer I've ever seen. I've seen undergrads working part time in the summer output 10x value to full time engineers with 5+ years of experience. "10 years of experience or 1 year 10 times" is the most accurate pattern to describe this.
Yea, it's a very different mentality to get yourself into. I find the system design question to be the most fun. Anyone can learn to code, anyone can learn your style guide/convention, etc. Very few people can be:
1. Given a problem ("{A client,we} want...")
2. Come up with a solution ("Ok, we'll be able to...")
3. Implement that solution with no oversight or assistance while only asking for pointers on where existing things are defined ("I'm trying to X, Y does X, where does that code/config live?")
I mostly work at small startups that need to improve their engineering quality so most of the roles I am attempting to fill are engineers who can be a massive value add and lead projects on their own without much hand holding.
If I was instead working in a company where 99% of the work was maintenance, phone calls, and busy work I'd probably forgo this screen-er. I'd likely not work for that kind of company but it's just something to keep in mind.
You can find all my past articles[1], and I'm happy to find you specific ones. I think, given that you can't switch away from algorithmic interviews, at least make the interviews more collaborative, define your expectations up front, and generally find ways to help the candidate show their strengths.
[0] https://hiringfor.tech/2020/02/10/false-positives-and-false-...
My ideal interview questions are as follows:
* No trick question. Reaching a solution shouldn't involve obscure knowledge or an "aha!" moment. * Only basic data structures and algorithms, the kind that you will realistically use in most day jobs. * Start with a very basic problem, and ask follow up questions with increasing complexity. * Extend the problem in different dimension to probe the candidate in different axis. For example, ask the candidate to design an API for the code, or to run it at massive scale.
It's hard to come up with problems that cover all these points, but you can get close. I find these kind of questions allow me to assess candidate without making the process unfair or overly stressful for them.
If you were a team lead tomorrow, how would you go about hiring your coworkers?
Honestly, that should tell you everything you need to know about:
a) What kind of technical skills they value (i.e. will work to improve in themselves if necessary)
b) Their interaction style with coworkers.
Sure, this interview can probably be gamed, but IMO a lot less than your typical "reverse a linked list" style interview.
Rather than saying how you are constrained, what freedom do you have?
1. Hire based on the resume, gut check on the interview. Is it plausible that they played the role that they said they did? If you were a major contributor to an RPC framework in C, but you can't walk a linked list... Maybe your resume is too inflated. This means that the questions are basic because their purpose is not to rank candidates but to ensure the validity of the resume.
2. Focus on reading over writing. I would rather a candidate who can read code someone else wrote, articulate what it does, and maybe plan for how to extend it than someone who can pretend to invent something new.
#2 I have mixed feelings about. it may make sense if you’re after an expert on that language but may evaluate wrongly candidates that would be great reading code once they get just a bit more familiar with and get the overall architecture. In many companies, you’d be forbidden from sharing actual code with non employees, so the excerpt shown in the interview would have to be purposely crafted and loose value. I propose a more interesting exercise could be to examine a random piece of opensource that Both the interviewer and interviewee are unfamiliar with and trying to get a good discussion from it, but it’s a risky move leaving the interviewer exposed.
A lot of interviewers get lost with these obscure algorithms, and forget what they're actually trying to test: the candidate's thought process.
A counterpoint on why I wouldn't ask anyone to implement something (such as a web server) using a framework: They're ever-changing. Furthermore, if I were to try and see if candidates understand building an API well, I'd rather ask them a mix of networking and open-ended, system design style questions.
Can you elaborate as to why this is a contradiction? Yes, they’re ever changing but doesn’t mean it’s not an interesting problem to solve.
As an interviewer, I don't really care if a candidate happened to know the answer to an algorithmic puzzle or has a deep knowledge of academic minutia. I want to know three things: can this person reliably and meaningfully solve technical problems, can they communicate their findings to the team, and finally – do they strive to learn the details or do they just skim/superficially solve problems? Obviously expertise in a job-relevant domain is great too - but I'd consider it a bonus; the difference between a senior and a junior/mid-level hire.
The first two are part of a quiz we give candidates, it has a couple basic questions on it that simply require them to logically think through a problem they likely haven't seen before and provide feedback. I don't even care if the answers are correct, but they should be able to explain their thought process and reasoning clearly in a couple followup questions. Bonus points to candidates that ask for feedback on their answer.
For the last point, during the interview I get the candidate to describe a project at a recent workplace, once they have given the description, I'll ask questions to see if they evaluate and learn from their previous work.
I am constantly shocked at how many candidates I interview that fail one or more parts of this spectacularly (especially senior level candidates), to the point where I question whether evaluating candidates these ways is unfair, but also knowing that if someone doesn't do this they likely are going to fail when given a problem without an obvious answer.
I have hired dozens of engineers and have never failed do find great hires.
What I mostly do is just a conversation, which I call “geek conversation”, just like when you meet people at meet-ups or tech conventions.
With a simple talk I can first make the person relax, this allows me to have a great impression about her/his personality, passions, interests, personal roadmap, attitude towards problem solving and etc. Whenever we ponder on a more technical subject I throw some questions that might help me measure their technical level a bit, but what I care to measure is mostly awareness on things, nowadays software engineers need to be, above all, great at researching, for most roles they must have a broader overall knowledge and be able to creatively join the dots with a ton of pragmatism. I don’t care if they are able to implementing Dijkstra's or to modify a Trie in front of me, although having awareness of what problem these things solve is a good indication of some seniority.
These interviews take a lot more from me, I have to properly learn about the candidate and the role before I enter the interview and that requires people skills and real interest in the product, and of course I am not able to interview 40 candidates to find just “the best one” (which is ridiculous and something I actually don’t believe actually exists).
Yes there are roles which require the extreme algorithm coding skill, but those are a minority and it is unfortunate if your company is treating every candidate as if they were going to do that.
Some people can. Some people can't.
The secret to a good hire is probably to exclude people from doing the interviewing who are essentially poor judges of character.
In other words let's stop trying to auto filter the candidates and instead let's filter the interviewers. Pick good interviewers.
Let those firms that play sunk cost Frat games get the sort of people who will work all hours for next to no pay. Those people always burn out quickly anyway. Instead treat the interviewees' time with great respect and you will get better long term people anyway.
I suspect saying that interviews are an hour's chat and then a decision will get you a set of outstanding interviewees who are just too busy to be bothered with anything else. And they will stand out from the crowd even on paper.
Most articles complain about the status quo but don't offer any practical advice. When they do, it is some approach that may not be an option on most cases - e.g. 40 minute interview followed by a 90 day probation period.
Live coding interview can be pretty tough for candidates, but we try to make a lit bit less difficult by:
- Giving them time and resources to prepare.
- Letting them use their preferred language and IDE/editor.
But since you have no control over format or time, maybe try to:
- Use reasonable problems - to assess skills people will actually need on the job. In our case it usually boils down to using comodity structures like a dictionary or map to do simple tasks efficiently.
- Use problems that start simple but can be extended. This is good because when candidates finish the first part, they get a boost in confidence. It is also nice because if they freeze, you can help them finish the current "level" and still have material to assess other skills.
- Set them up for success during interview:
- Reserve the first 5~10 minutes for introductions, ice-breakers or questions about the interview.
- Reserve the last 5~10 minutes for the candidate to ask any question about the company or the hiring process.
- Make clear that is OK to ask any question.
- Help them on small blockers.
- Be friendly.
Some links/references:
[1]: https://lawler.io/scrivings/erics-guide-to-hiring-software-d...
[2]: https://medium.com/@alexallain/what-ive-learned-interviewing...
[3]: https://www.holloway.com/g/technical-recruiting-hiring/about
Of course, this requires knowing what skills & knowledge are needed for the role - you may need to review the job description or push on the hiring manager to quantify them.
When asking problem solving or coding questions, work with the candidate as if you were two teammates solving the problem together - don't be adversarial, don't play "gotcha!", don't try to show off or prove yourself smarter.
If you have an awesome candidate, they will be smarter/more skilled/more experienced/more knowledgeable than you in some technical areas. That's a good thing!
- algorithms
- experience with a specific technology
- getting things done
- communication in "not being a dick" sense
- generally "being smart"
I want a person to score "B" in all of them and "A" in one.
It's important to realize that you don't need all A's: it's good if you manage to find an all-round A-student, but it's rarely possible and takes huge amount of time.
One "C" may be allowed, but then I'd probably want more than one "A".
So I keep my questions simple. For algorithms, for example, I ask to write a bubble sort. If a person can do it - it's a firm "B".
I generally take a look at their resume and then do some research about the tools they've used in advance. Then, during the interview, I ask about what they like/don't like/find interesting about those tools. The goal is absolutely not to gotcha them, but instead to find out what they're interested in in that space. If I ask a question that it becomes clear they've lied/fabricated about on their resume, I say something to the effect of "No worries" and change the subject.
Depending on the role, you need more info than just what languages/frameworks they've used. For more senior roles, or roles that involve architecture/cloud functionality, I'd ask about how they've built systems in the past. If they call out AWS, I ask about what resources they've used, how, and why. If you've written down DynamoDB but cannot speak intelligently about access patterns or secondary indexing, it's kinda clear that you just used a system someone else defined. Whether or not that's a problem depends on what role they're applying for. If they can speak intelligently about how they got to a specific DynamoDB structure, they probably are being honest enough about their experience. Note, it needs to be clear that the candidate is not speaking in the abstract, but about things they've actually done. Googling stuff is easy, finding the weird parts of tech in practice is hard.
Ultimately I want them to feel comfortable enough to get chatty about development. Usually I find out enough about their skills while they're chatting - I think most would be surprised to find how clearly you can understand a person's abilities without directly asking about them. You just kinda have to spend some time up front learning pros/cons/common pitfalls of the tech on their resume.
There's always a ton of posts, but I have never had a pleasant or unbiased interview experience in my life.
It shows to the interviewer how a given person works in a programming job, how they researched solutions to the problems they were faced with in the past etc.; as opposed to the usual exercises that have nothing to do with the job and won't really show you how well they will perform.
It's also less confrontational which should put the interviewee at more ease.
So what I started was a local library program (pre-covid) that I would attend before / after work to be around people who are doing career switch to coding.
Then when students learn enough and have strong foundations that I look for, I recruit them to work on open source side projects that benefit new students who are learning.
For the students who have shown the maturity to build and maintain production features, I refer them to the company I work at and they interview and (hopefully get in).
In the past 2 years, I've referred 5 students successfully and they're all doing pretty really well (I secretly stalk their internal code contributions).
I hope that one day, I could have enough reputation at my company to hire engineers based on their open source contributions. I also hope that the stock market doesn't crash next year because I plan to use my RSUs to start a fund to hire junior engineers students so they don't worry about medical benefits while they contribute to open source projects and have a hard time finding a job.
If you mentor a student as they are learning, you can instill good engineering practices: like taking the time to communicate and write good documentation, write code and comments to make life easy for the next engineer to take over, how to handle tradeoffs between time and quality, etc. These are critical engineering skills that are very hard to test in 1 hours but are very effective if mentored early on.
I find it astonishing how many developers absolutely cannot think through a tree model. I deal with tree models absolutely every day in programming, management, and personal concerns. For example: the DOM, file system, company/employee organization, outlines, security models, business/customer needs, and so forth. All these things are tree graphs and I encounter many of these EVERYDAY.
Many developers simply cannot explain walking from one point on the tree to another point. I am not even talking about code. Even in casual conversation I am astonished at how many times I have encountered people like lawyers, QA, bosses, and others who can engage this sort of logic while some of my developer peers are just lost on the subject.
That is absolutely something I tested candidates for as a filter and will continue to do so. If a person even talking through such a real world common form of abstract logic how can I expect them to write code? It’s like they expect some magic wand to simply provide the logic for them and just sprinkle on a little bit of syntax.
1. Tell me briefly about your professional development, including challenges you have faced, mistakes you have made, how you overcame them and what you have learned. Have there been any critical mentors or resources? What motivated you in each position and how did that lead to your current application.
Opens a floodgate of possibilities for discussion, generally gives a sense of career pattern or path, potentially personality.
2. What are you learning right now and why?
Screens for curiosity and/or goal orientation.
3. Choose any recent technical or business idea you have had and explain why you considered it to be interesting.
Possible insight in to how prolific their idea generation may be, and whether it is circumscribed by professional domain or not.
With a bit of two-way questioning on these, you should have a basic sense of their capacity as a communicator as well as personality.
Important: Predominantly co-workers and direct managers should interview, not random HR departments, external contractors or disconnected tiers of management.
We work on fixing that, so we implement a process: founders pick some personality traits they find essential for a great hire, we prepare an interview playbook, and they can be certain that the new guy is actually what they are hoping for.
Interviewing for personality (cultural fit) is hard because we (humans) are hardwired to survive so we give answers that we think are in our favour - and not necessarily truthful. Key to our approach is that we use falsifiable questions, meaning that you can't guess what I believe is a right answer.
If this helps any fellow founders here, am happy to lend a hand.
Now pick some arbitrary rating system (5 stars, school grades, 1-10 points, whatever). It does not matter that much which one you pick. The hard part is to stay consistent about it over time.
Now, here is the way how to make it more pleasant for your candidates. After 3/4 of the time, write down your rating and discuss it with the candidate. E.g. "I gave you four stars for cultural fit. I get good vibes with you but you would be the only PhD in our startup, so you are probably more academic than everybody else." Give them a chance to correct your rating. "Oh, I forgot to mention I'm involved in my brothers construction startup. I feel fine in that non-academic world."
Giving such honest and clear feedback to candidates is a better experience than most companies.
How would you design/write xyz
What's wrong with this code [code] (bug, security issue, bad code)
What do you think about
Use multiple (2-4) recruiters paid on contingency. Receive about 5-20 resumes a week from them (total, not per recruiter!)
Filter these resumes to about 3 per week. Interview these three yourself for 30 mins each. Just ask them for more details about technical experience on their resumes to detect liars. If your recruiters are any good they will have filtered out people based on the soft skills / career questions.
Reject the liars and accept just one of the honest people (if there are any). Push the one forward to your team. Let your team loose on the candidate with any bizarre (but legal) questions they like. Give them the final say on the hire. Repeat until position is filled.
I never focus on algorithms or particular technologies. When I interview someone, I'm looking for two things:
- First and most important: we both are in the same wave. I want to make sure the applicant understands what's the goal of the position. For example, once a guy with over 10+ of experience applied for a junior position but demanding responsibilities and salary of a senior.
- Second, expose a general problem and work with the applicant to get an idea about what can we do. I don't expect a solution in 5 minutes, I'm looking for ideas of how we can tackle this problem and how we would start working on it.
2 what would be a moonshot feature for P? (If you had a perfect AI lib or unlimited processing power or devices spread all over the world)
3 if you had to find someone to take over your job (for you to manage the new one), what would you look for?
-Don't ask questions designed specifically to gauge how the candidate 'thinks'. If you do pose such a question have another interview with the candidate some days later so that the candidate can think though various solutions.
- Don't ask questions which are cliched like 1) Tell me about yourself 2) Tell me your strengths and weaknesses
-Don't continue interviewing a candidate if you have decided not to hire him/her.
What you can do:
- Ask them to look up an answers using google/search
- Paid assignments
You discuss different architecture options, feature trade offs...
It's easier for both parts to connect, it will give you a glimpse of how the candidate faces requirements from other teams and you give him a taste of the inner workings.
It was an easy assignment that covered a lot of concepts, and it was followed by a 2 hour techinical discussion on it and all things related and unrelated with my future colleagues.
Outside of that the interview stuff was just around personality and culture fit.
The algo interviews are partially to test can you do the stuff. If you have done similar and can describe it in sufficient detail, then you know it.
First couple of rules:
- quick turnaround. We want to give an answer to a candidate within a week they engage us. If we make an offer, we give them plenty time to sign the offer, even shop around, before they make the decision
- no step can be repeated or returned to. If a candidate progressed from step 2 to 3, our hiring managers are not allowed to go back to the previous step
- we disclose as much information as possible early so there are no surprises
Here is the process:
1) Initial 30 min call with one of our recruiters. We cover mostly the company, the compensation package for the role (usually range based on the level they fit in) and our expectations. We also cover the culture and share as much as we can so the candidate can make informed decision if this is a right fit. We ask some high level questions, but usually focus on the company itself.
2) Take home challenge. We have one for every type of position, from engineering to marketing. They are usually simple and fairly arbitrary problems that illustrate candidate's thinking about problem, how they analyze it, what tools they choose to solve it, etc. When we design the challenges, we are aiming for it to take around 30 minutes. We are not trying to get the person to any significant work for us, rather just figure out how would they solve problem if it was given to them at their position.
3) The candidate meets 4 people:
- their manager where they cover the role, the expectations, the growth at the company and anything related to their job
- random colleague. Essentially someone they will not work directly with. We use this person to help us to focus on the candidate themselves rather than their skills
- our COO, who covers with them more high level topics like what they did/didn't like about previous employments, what they care about in working environment and such
- our CEO (me). I usually focus on their individualities, especially passions and hobbies. I want to see that they care about something, anything. I also answer any unanswered questions from vision, company's cash position, customers, how we generally work, etc
As I mentioned, we try to get an offer to candidate's hands within a week. With this process we get around 95% offer acceptance rate.