VP of Engineering at Eaze, previously CTO at Getable and engineer at Yammer.
No, not at all. Whiteboard interviews test one thing
well: How well does a candidate code on a whiteboard.
Engineers on my team never have to code on a whiteboard
(whiteboards are really bad at running code), why would I
make candidates do something that I don't ask the
engineers already on my team to do?
What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
I hate this concept that if it's not typing code into an editor then it's not "real work".
I absolutely communicate with my coworkers using a whiteboard and pseudocode. I reject the idea that being put on the spot is necessarily "artificial". To the contrary, I think that the number of engineering jobs where you can assume you'll never be put on the spot or have to communicate complex ideas verbally or visually is relatively low.
Now if this particular skill isn't interesting to an employer and they'd rather spend the time talking about some other candidate capability, that's a whole different story.
Whiteboard interviews rarely test the ability to communicate on a whiteboard because the interviewer knows the answer they are looking for and the presenter does not.
If you defend whiteboard leetcode using this excuse you are deluding yourself and you are part of the problem. The only thing the modern FANG interview hires for is people that can solve algorithms problems in a silo under pressure. No checks for software engineering skills, collaboration skills, testing skills, reviewing skills, and on and on...
The whiteboard interview is why so many engineers unexpectedly suck.
I think it's safe to say that being put on the spot in front of people you don't know who are judging you is quite different from being put on the spot in a team meeting with someone from product who will watch your demo of a potential new project/idea/concept.
If you want to test communication skills amongst programmers, I dunno...ask them to write something in coherent English. Or here's a crazy thought: ask them to document some code. I guarantee that 80% of working "rockstar coders" will fail (but not before whining, crankily, about how unfair it is that you would actually make them do such a useless thing, since, y'know...never actually documents anything IRL, dude).
Programmers like whiteboarding because it lets them believe that interviews can be reduced to purely objective functions, and because GooAmaSoftBook does it. They're too scared to deviate from the pack, lest people think they aren't as "elite" as everyone else.
Perhaps it would be more effective to have the candidate whiteboard a concept that they are already familiar with, be it a high-level engineering principle or a system/solution they have built in the past. Attempting to solve a problem you have just been presented with AND communicating the solution effectively is a big ask.
thinking and presenting are two separate skills, and, in my opinion, can rarely be executed at the same time.
What you're testing is the ability to discuss a problem with your peers. That is the critical thing.
Do the whiteboard but don't make the coding the standard. Don't get too hung up on the details of the problem.
Make the problem easy but fuzzily specified.
Clear and respectful communication is one of the most important skills for an engineer to have.
If you read the second half of my answer in the article I go into the communication side of things. In my experience there are better ways to evaluate a candidate's emotional intelligence and social skills than asking them to code in front of someone.
In that case I would recommend reading the candidate's resume beforehand, look for prior experience publishing, presenting, or teaching and ask them about it.
Do write lots of sequence diagrams, block diagrams, ui sketches, arch diagrams and very occasionally someone might draw a data structure.
Whiteboard interviews where the interviewer says "it's actually `.toLowerCase()`, not `.lowerCase()`, you fool!", or "oops! You forgot to `position: relative;` the containing element that you `position: absolute;`d below!". you're not testing anything useful. Maybe if we were still coding everything on punchcards and using massive user manuals, but docking applicants for things their editor would highlight or one ctrl-r on the page would find is ridiculous.
This is incredibly common, and you tell the candidate in advance that they will be expected to give an n-minute presentation on a subject of their choosing (need not even be tech) so they can prepare.
Tell me this: when did a mob last ambush you at your desk and demand that you invert a red-black tree on a whiteboard that they happen to have with them right now? I'm guessing that never happens where you work. If it does then fair enough :-)
Ironically this leads to min- maxxers who ignore a good chunk of their courses simply to master leetcode. My smartest peers are ones who work on interesting projects and research, thus never finding time to grind interview questions.
Then there are also companies that expect your white board code to compile and have perfect syntax, which imo is very unreasonable.
You're 100% correct. We give our candidates the choice between working through a problem at the office (on the spot as you say) or doing a take home project asynchronously. Not everyone has the time to do homework, but not everyone wants to do an in person coding interview either. We try to stay flexible.
> I am a quote
This makes the quotation doubly-distinct, and still readable on mobile devices.
===
To the actual point I wanted to make: plenty of engineers at companies code on a whiteboard. They schedule a meeting, grab a room, and talk things out, while making notes, diagrams and sometimes even actually writing out some code on a whiteboard.
I hope nobody asks people to write compilable code on a whiteboard, but asking a person to talk through a problem and jot down some pseudocode doesn't seem like an awful thing to me. Communication is a pretty critical part of our job.
Not always a whiteboard, either; one of the best engineers I ever knew flunked a Google interview because of a situation where he had to write a bash script and read it over the phone to the interviewer, who apparently didn't transcribe it correctly on the other end. But "they only hire the best", you know, and their practices are rock-solid and proven.
I've _never_ seen someone, or have asked to participate in, some sort of whiteboard coding thing. Even with pseudo code. That just doesn't seem helpful to me, at all.
Actual Code? Not a chance. Portability, Durability, Efficiency - It's the worst of all worlds.
No comment on it being an effective interview process.
And interviews are not pressuring, they are simple. No hard desisions, no anxiety whether it is going to work. You in and 5 hours later you out. Lame.
an advice to candidate: when you've been asked to do whiteboard coding - run away. this is a negative signal.
During my recent job search I was asked many questions I didn't know the secret algorithm to, but if those companies had given me the problem ahead of time and allowed me to research it -- just like what happens in real life -- I could have coded solutions to all of them.
And if you're worried about people just plagiarizing existing solutions, have them talk through it, as you say. Should be fairly clear if they understand each line or not.
1) Whiteboard
2) Review and technical discussion of existing code or projects
3) Paid take-home project
4) Pair programming on live code
5) Technical discussion only for rare cases for those Top Secret/NDA candidates that really can't discuss their past projects but are also time constrained
Some people do actually like to whiteboard. Keeping it around is kind of like legacy support for people who practiced hard to excel in those interviews.
This is pretty much the union of technical interviews that are well-known. If you switch out a technical interview format and can't get meaningful insights into a candidate, you're testing for the wrong things.
Most of what we do as engineers is trying to read code to figure out what it's doing, how to fix it and how to enhance it. Going through real code with a candidate and assessing their ability to understand brand new code (and what questions they ask) seems like it would give good insight to how they would perform on the job.
I've been on dozens of interviews, and that's the only time I've ever encountered that, but I think it does a really good job representing the job, as that's invariably what I'm going to have to do when you hire me, is sit down and familiarize myself with the code base.
He also asked me a handful of questions, said "Okay I know you know enough to do the job, let's see how much you really know," asked me a bunch of really low level stuff, and proceeded to teach me concepts when I told him I didn't know certain things. I walked out of that interview knowing more than when I walked in, something else that has never happened in all the other interviews I've had (except maybe I learned a new term that I never heard before that I was apparently supposed to regurgitate back because that's what was written down as the answer on HR's answer sheet).
Most whiteboard questions are always some esoteric algorithm any engineer worth their salt would in real life would look up before ever consider using to verify their own understanding before using it.
Putting someone on the spot saying whiteboard x is not ok.
However if the test is to see if they can take part in a meeting, then by all means do a white board, but ask them draw a workflow diagram or some other 40,000 foot type of diagram. Do not ask them to put code on a whiteboard.
(and if you wonder how anyone ever gets hired there, the answer is: by cheating. If you're going to interview at one of those places, odds are their interview problem and an accepted solution will be online somewhere, so you just go look up and memorize)
So much of the technical hiring process tends to be about trying to find people that match what you expect people to be. Which might be fine, you need to fill a particular position which requires a specific set of skills. You need to validate those skills. You do that by doing something semi realistic while trying to minimize the pressure of an interview situation.
But, then there is the other kind of approach where you are looking to see what the candidate can offer, trying to find people who know different things to you and do things differently than you, in this scenario you are more inquisitive and let the candidate direct the interview process and get them to show you things that sound interesting (including coding).
The first approach is all about finding specific skills. The second approach is all about being clear about the values you are looking for.
The second approach requires more skill and more time while wearing the hat of a technical recruiter.
You may need a little bit of both approaches.
So the big thing is to be clear about who you want to work with at the end of the process.
You end up spending all your time dicking around with missed semicolons and looking up library routines and shit like that. Sure, it's all stuff you have to do as part of a job, but it's also all trivial stuff.
The last time I had to do this, I spent time googling how to open up a port in python, and then read from stdin. It's just a waste of time.
All this does is test the wrong things.
Here's my attempt: https://youtu.be/6LHqrxrC6Uo
I solved the problem, with an IDE and a compiler. I communicate during the entire exercise. It takes me over 2 hours. Granted, I debug it, so that takes extra time. (Can't debug on a whiteboard!) I had to take one break. On top of this, I've solved this kind of problem before and it still took me more than the 45 minutes you're likely to get in a whiteboard interview.
And I try to cover all of the "hot topic" points I know the interviewer is thinking about: size .vs. time tradeoffs, iterative .vs. recursive, etc.
If I had to solve this problem on a whiteboard, in 45-60 minutes, with the bar being "it must run"....I would fail.
I guess that makes me a poor programmer. Instead, I'm a grumpy old man who is really tired of all this fraternity hazing bullshit.
I had to solve almost the exact same problem very recently. I find the best way to start is to just implement the easiest solution you can think of, such as the array approach you mentioned in the beginning. I did that in 5 - 10 minutes. We spent the rest of the interview optimizing and fleshing out the problem (e.g. let's not use an array, but there is only one "color").
Overthinking the problem straight from the beginning and potentially being unable to provide a working solution at all is probably the worst thing you can do. I actually had that happen in a previous stage of the aforemention interview, and had to "reset" by going back and implementing the simple solution before optimizing it.
Probably you got exhausted after doing all these preparations and debugging issues not related directly to the algorithm. I'm not sure the final solution is correct, it is not tested on a graph with multiple connected segments of the same color. It looks more like a convoluted way to count colors globally, discarding standalone colors, if I understood your solution properly.
Sure you COULD use a graph, but using a different code abstraction to represent the idea of a graph is much more pragmattic and maintainable, of course that varies with application.
Im reentering interviewing phase right now, and its funny revisiting these problems with a new lense. The mind is very plastic and its exciting, so far as things to come.
Total nerd out moment :v
Of course most people prefer to interview in a language which is less verbose than C++, which saves time.
You also keep a "counted" and "visited" variable separately for no reason. There is no reason to visit a node and not count it, and you can't count a node you didn't visit. Whatever optimization you think you are getting surely isn't worth it.
As an experienced and good programmer candidate, the most important part of an interview is for you to evaluate the company. So the whiteboard is a great way to see if they understand what can be accomplished in 15 minutes on the whiteboard.
If they don't get it, you can end the interview short. Time saved today as well as the next several years working for a shite company.
I love the whiteboard interview.
I explicitly described the conditions of the original interview. The interviewer asked the candidate to "solve it" and it had to run. What part of that wasn't clear?
I also find it interesting that you countered with a completely different kind of whiteboard scenario. I agree that the scenario you describe is fine. I would enjoy those kinds of interviews all day and twice on Sunday. I'm not railing against an honest discussion with a potential employer. But that's not what happens most of the time, in my experience.
Also, the tone of your reply is that you reject the employer if they don't meet your definition of reasonable. That's great, but not everyone has that luxury.
* Whiteboard coding is awesome for software development shops that don't actually own computers or use punch cards to load programs.
* Shared coding environments with a time limit works well when you want to double screen for someone who can also diffuse suspicious packages that arrive at the office
* Riddles and puzzles are good when your workplace has a chicken coop, an office fox and you can't figure out how everyone can go for lunch without leaving the fox alone.
* Behavioral interviews are appreciated by candidates who took Psychology 100 back in school for an easy A.
* Take-home tests go over well with the huge population of talented software developers who can't find a job and have loads of time they want to spend decoding the operational cost of a bubble sort
The only thing I've seen that's at all realistic and effective is a very short, __paid__ project. I did a two-day one for Indeed (ironically one of the worst promoters of all this bullshit) and a 4-hour one for my current employer. The payment doesn't even have to be market, it's more important as a signal to candidates that the company values your time.
I ask for 3 things from perspective employers:
1. value my time like you value your own (both the number and composition of your interview steps)
2. keep me updated as to where we're at in the process and when the next stage/decision will be made
3. Move forward in the process in a timely manner. It should not take more than 2 weeks from when you initially contact me to the process concludes.
I've never gotten more than two of the above from a single organization, but I will someday, and I'm betting that a company that treats potential employees that well will treat actual employees better as well.
It's so simple to figure out if you're on your own and just play around with the idea. When asked on the spot I just kept thinking "hey these people want an answer NOW, don't make them wait" while feeling their stare, and couldn't figure it out without help.
t0:{[3,5],[0,0]},
t1:{[3,5],[0,5]},
t2:{[3,5],[3,2]},
t3:{[3,5],[0,2]},
t4:{[3,5],[2,0]},
t5:{[3,5],[2,5]},
t6:{[3,5],[3,4]},
}
Thinking about making this an algorithm, are we forced to use recursion?
If it's very short, then almost by definition it's useless.
That being said I've had great results from doing, say, a tiny consulting project, and ending up with full time employment. The best jobs of my life have followed the pattern of "am either offered, or I manage to ask/negotiate for a small consulting project" -> "before the project actually ends, am brought on full time".
My time is valuable. I see no reason to work on a useless task for free.
* Paid projects are appreciated by candidates who are currently unemployed.
They are all imperfect proxies due to various constraints.
I wish this project approuch was more common. It would probably be cheaper than wasting time on multiple days interviewing reality shows where people are voted out one by one like some companies do it.
True story.
That hasn't stopped it from showing up in every programming interview I've ever done.
A question I've had in an interview was what is the difference between recursion and iteration. Fairly easy to explain, including examples of usage.
White board coding interview are testing only one thing: how well candidate is prepared for it. And here what is wrong with it: The assumption that candidate should spend his/her valuable time preparing for someone's assessment is arrogant. I do understand why Google and Facebook do it (the do other arrogant things), they assume you want to work for them so much you will study to make the cut. So, they track bunch of metrics, which makes THEIR life easier. They have baselines, calibrations, and other things you never have in the real life. Hilarious part is that they are both risk averse (better to not hire a good candidate than hire a bad one) and using brood-force approach (they will interview you endless number of time).
And if you are not prepared - you will most probably fail, unless you are really good at it or lucky. So, why should I prepare for Google's or someone's else interview? I have a interesting and very busy job, I just don't have time for this. I have commitments to my team to work on real products, not on fake problems from Cracking Google Coding Interview.
Just think how absurd it is?
So, starting from 6 month ago I: 1) Tell recruiters upfront I will not be spending a minute preparing for their interview process 2) I decline coding on the whiteboard. High level designs are fine, writing code - no, no exceptions 3) And the most important, I stopped asking candidates to code on the white board.
The most important skill of the software engineer is (IMO): ability to keep complex problems in context for long time (not 5 boxes and months), ability to manipulate them and ability to convince others that you idea is worth pursuing.
Because you want the job? Everyone is busy, that's a terrible excuse not to brush up on your interview skills.
By the tone of your post, my sense is that prior to your recent epiphany you weren't very kind to candidates who you interviewed.
>> Everyone is busy, that's a terrible excuse not to brush up on your interview skills. I believe that their approach is arrogant, they just don't value candidates time.
I have a confession to make, I spent some time at Google. And after I left I was interviewing people using their approach and I regret it now.
P/S/ When you interview for a company, you actually taking more risk that the company (talking about folks who are headhunted into interviews). If it doesn't work out, you will be out searching for a job, you career growth may be slowed down. And what is the company risk?
Plus, if some one has to 'prepare' for an interview that is supposed to test people's capabilities to do 'everyday' jobs, then you are hiring the wrongest possible people out there.
It's about the thought process when breaking down a problem and the ability to organize and communicate their thoughts. It's also about exploring a problem space. How carefully does an engineer consider edge cases? What kinds of things are important to them?
A whiteboard to me is a much simpler and accessible medium through which to explore a problem vs setting up an environment and having to type code and dealing with all the minutiae that comes with actual development.
Of course, it isn't the ultimate method of evaluating a candidate but it's certainly valuable.
Isn't that a more realistic scenario? How often do you give an (employed) engineer a task then ask them to immediately explain what they'll do to implement it?
If, however, the job requires careful consideration of dependencies, time domain, and, yes, edge cases - then you are NOT testing for that.
Lock the candidate in a room and give a few hours to let her to produce the result would be a better measurement.
Lots of interviewers suck. That isn't a property of whiteboard interviews.
Just give people a search bar and ask them to solve a problem outside their comfort zone. See what terms they use and how quickly they can arrive at a useful link. No pseudo-code or architectural diagrams needed. Just a link with a plausible solution. Because that's how most learning on the job actually happens.
Anecdotal: we had some problem to solve which required qua a lot of code to solve and only old sites with broken links came up in Google for the guy solving it so he decided to implement it himself. When I asked why that task was taking so long he showed me what he was implementing. The code he was writing had an implementation on an old site which had a broken link to the code; when I copied the name of the zip file that was linked in github; there was the code. It had no description or comments; someone just checked it in. It was not perfect code, but it shaved off a few days of writing it from scratch anyway.
I care about being able to code, which can be assessed from past code and a pair programming session and then efficient problem solving using your brain and internet. The latter is somehow less common.
So my favorite way now is; I ask them to 'solve' a problem they have never encountered and that cannot physically be done during the interview time (I am interviewing for software development and in our field you get asked for things that cannot be done in the time you are asked to do it all the time, so I need to see how you handle that); give them time to Google possible ways to do it and present a little plan. Then they need to pick, out of the plan, something they think they can do inside the interview time (see how confident they are and if they have any sense about low hanging fruit or estimations; I might suggest something else if the task they pick is only installing libraries for an hour ;) and we'll sit together behind a monitor and do it together. I'll usually tell them to stop after 15-30 minutes as I have seen enough at that time.
Additionally, just try to look up items on a language that is in (constant?) flux like Swift. Swift 1.x? 2.x? 3.x? 4.x? What's the API today -- naming convention changed, parameters changed, etc. All I can say is that thank goodness Google allows for date ranges with searches. :-)
I've been doing this for 15+ years professionally, have a master's in CS and I struggled to answer many questions. Just stood there blinking like a goldfish.
If companies really wanted to simulate programming work, you'd give me 10-30 minutes to Google the problem because THAT'S HOW IT WORKS IN REAL LIFE. Which is how it should be; rarely are you solving a new problem, usually other people have faced it before, it's probably far more efficient to start with a known working solution and iterate from there.
Look, of course there things you should know. Data structures, algorithm basics, some complexity theory, etc. But asking me to output numbers in a matrix in a spiral fashion does not translate into real-world ability.
I find the more important developer skills are people skills and project management skills. How well do you work with others? How do you resolve conflicts? What's your methodology for grouping work-- Jira tickets, sprints, etc?
But almost none of the interviewers seemed to care. They wanted to know what exotic technology skills I had, not that I could work with project managers and marketers effectively.
Take-home projects are more time-consuming but the least unfair, whereas whiteboard coding is the shortest but most unfair.
I blame Google for this curse. I hope the next time I go looking for a job this process has changed.
Ask me about past projects. Want GitHub examples? Fine. But whiteboard coding some non-real-world problem?
I get that companies need some sort of filter, but whiteboard coding is not the answer, IMHO.
I've worked with folks with 15+ years that could technically code but they took eons to deliver and their solutions were unmaintainable. Some of them with advanced degrees too.
Small sample size, but the few folks I'm thinking of were such bad hires that could have been avoided if we had them write some code or put the proper emphasis on the coding exercise we had them do.
The only time I've used whiteboards in my job is either when I need to hash out a solution or approach with my coworkers, because I'm confused or don't understand the problem, or when I need to understand high-level architecture for a system. It's a tool for clearing one's thoughts and for getting a team "on the same page".
If you're hiring someone to code-monkey a bunch of JIRA bugs, then whiteboarding might not be a relevant test for them. If you are hiring someone to build out new features efficiently or redesign architecture, then whiteboarding may be useful for having them describe at a high-level how they would solve problems. In both cases, writing syntactically flawless solutions on a whiteboard is a waste of time for both the interviewer and interviewee.
I say this as someone who worked at one and interviewed at others. You really do need to treat it like an exam.
1. Have a senior engineer chat with the candidate for an hour about a few bugs/features in the repository, and how the candidate would solve them.
2. If the candidate is promising, give the candidate a week or so to fix the bugs or implement the features.
I think this would be a more relevant test than just giving them context-less problems to solve on a whiteboard.
It would also provide a significant signal regarding valuing the candidate's time and effort [1], which would comply with the suggestion of another comment elsewhere in the thread.
[1] As well as the additional signal of putting money, rather than just "free" candidate labor into open source.
Here's the thing though. Let's say you're the one giving the interview to two people who claim to be able to get shit done. One is able to code up a solution quickly and the other struggles, producing less, or producing something that's flawed. Sure, that second person might actually be productive under the right circumstances, but that doesn't matter because the first person already proved they can code.
That's the whole point of interviews, getting high signal in a short period of time. If you think coding interviews don't provide high signal than propose something better that costs the same amount of time.
The whiteboard is not providing "high signal". Everything else is. The whiteboard is providing low signal, and taking quite a bit longer to do it.
If that person studied much more, then they weren't really "quicker", they were just better prepared.
Perhaps that's still a reason to hire them, but not everyone has time to study lots of programming puzzles before an interview, and this is then less a measurement of speed, and more of preparedness.
It also doesn't take into consideration whether the whiteboard problem being presented is at all relevant to the job being offered.
So yeah.
Currently, many interviewers go for - as I like to call it -"trivia questions". Personally, it just doesn't seem appropriate because "Everything may be obvious, once you know the answer". In some such interviews, I've asked my own set of trivia questions to the interviewers. The interviewers weren't pleased as they couldn't answer some questions for which I knew the answers beforehand.
For example, one I like to ask (in person, with no computer access), ostensibly to gauge familiarity/intimacy with the Unix CLI, is:
"For each letter of the alphabet, see if you can name a Unix command (anything one can type at a shell prompt that won't return a "command not found error") that starts with that letter."
I preface the question with something suggesting that it's supposed to be fun, and I encourage skipping letters and coming back to them to avoid pressure (nor do I write down the answers). Most people are surprised at how many they remember without much thought.
Although not particularly deep, further discussions can arise from differences between shells with respect to built-in commands, as well as differences between OS flavors. Also, I've discussed how a somewhat new language like Java has made one of the otherwise difficult letters much easier.
Regardless of the trivial nature of the question/exercise, it provides a spectrum of jumping-off points for deeper knowledge discussions, all of which are signaled/chosen by the candidate.
Have someone else pick a coding problem (that you as the interviewer don't know beforehand), and work on it from scratch together with the candidate.
Being upfront with the candidate that you don't know the answer yourself puts the candidate at ease, and you'll be able to better judge the candidate's soft skills (communication, critical thinking, empathy in case I don't understand the problem).
Most of the problem seems to me fundamentally unaddressable without verifiable portfolios, but those have their own problems. Even in industries where portfolios are standard, plagiarism is a thing - how do you get the verifiable part? I want to hire someone great, not just someone who was on the same team as someone great.
That general premise is right, we've lumped too much into the term "whiteboarding", and it's worth unpacking into the different interviewing practices we may like or dislike. Some things people don't like have nothing to do with an actual whiteboard (e.g. testing esoteric CS knowledge or deliberately creating a stressful environments), and some things that involve a whiteboard IMO are ok (e.g. sketching out a system diagram).
That is STILL a thing (judging by HN) and it's useless. I don't think anyone would ever think it's a bad idea to have a whiteboard on hand if a candidate is asked to describe how they would design some nontrivial system. Being asked to describe something without a whiteboard in that case, is even harder. The objection you often get for the latter kind is that it tends to benefit the extroverted kind that likes to babble and sketch, whereas a candidate who would prefer ten minutes alone with a pen and paper might not do so well. I think are some merits to that complaint, but I also know from experience that being able to babble and sketch is really important.
If you want to see how well someone codes, look at their code. Or ask them to write something for you. Don't do a whiteboard interview and think it is the end all be all. Take it as a data point.
If that were the case, ask me about a past problem and my solution and to defend it, not the proper way to invert a b-tree.
I don't think design interview alone is that useful either. Someone without actual experience designing stuff could still fake it by reading other people's past solutions and answers, there are templates on the internet.
Nothing is inherently bad just because it fails you. Everyone saying Google's process is broken, blahblah, doesn't change the fact Google employees are hot AF on the job market.
You can if you're willing to work for a smaller, non-famous company.
To me, whiteboard interviewing makes sense when hiring for a role that leans toward a comp-sci background. People who do well in these roles typically have a brain that works in a certain way.
The roles that lean away from a comp-sci background are usually the type of developers who don't do well with whiteboard interviewing techniques.
What too often happens is that a hiring engineer doesn't interview within the context of the role they are hiring for.
You know this how? Intuition?
People have all different kinds of backgrounds and brains.
When I say simple, I mean really simple, like FizzBuzz simple.
There are interesting questions you can ask that are really simple which are not FizzBuzz. I've used the Fisher-Yates shuffle algorithm and more recently Selection Sort. I should stress in both of these cases that I carefully explained the algorithm before asking the interviewee to code it up. Even though these algorithms are quite simple, I do not expect the interviewee to have memorized them.
I do expect someone who is a professional software engineer to understand the algorithms after a decent explanation and then write code which will implement them.
I also don't necessarily expect the interviewee to get the code exactly right. I just expect them to demonstrate that they could write correct code given enough time and access to an actual computer to test the code on.
Once they hit the on site interview it should be more about how they can work together with a team, how they would design systems, what they would do if they needed help, etc.
Coding up FizzBuzz on-site is a waste of everyone's time.
Once in the workplace however I usually get promoted up pretty quickly. I'm pretty handy with generics and meta programming along with architecture and design patterns, meaning I can turn out more complex systems in much less time than most of my coworkers. But that crap never comes up in the google/microsoft interviews of the world.
If you do have a CS degree, then I expect you to be able to show decent competency with that material because you were sitting in all those classes for all those years. If you didn't learn anything, it makes me question whether you apply yourself to what you do or just coast through stuff and are going to coast through at this job too. Since we have the shared experience of a CS degree, if I understand what you got out of it, I think it tells me something about your attitude and approach to your work.
If you don't have a CS degree, then I am going to focus more on skills required for the job role itself. If being strong on algorithms is really necessary for the job, then I'll need to cover that. For any job, though, I'll need to convince myself you have a certain base level of competency so that, for example, you know that a linear search of a billion items is bad.
Speaking for myself, I like pair programming interviews.
When we view the hiring process in the context of what it actually is, a sales relationship, we realize that the customer (i.e. the employer), is ideally looking to buy services and time from the business (i.e. the employee/contractor). Viewed in this context, we quickly realize that many different things become part of the equation. Think about hiring a contractor for your home. If you happen to understand the job they do, you might be able to assess a the potential contractors previous work, quote and reputation effectively. Most people can't because they generally don't understand what it might take to do a job and have no experience in executing it even if they do. So you either heavily rely on recommendations from others, or go for a lower or higher priced contractor based on your perceived value (cheaper job, w/e the results vs more expensive job and risk mitigation) and hope for the best.
Considering most software development doesn't actually happen at "tech" companies, the reality is most companies are like most people and likely can't properly assess talent no matter what. They essentially get lucky if they get a good candidate or unlucky if they don't. That would lead us to think that maybe the "tech" companies are able to do a better job of it. In essence they do, but not because of their methods. I'd say it's a form of selection bias.
Google became successful and happened to be run by generally smart people technically capable people. As such they were able to attract generally smart technically capable people to work for Google. I'd venture a guess that 90% of people who make it to the live interview stage at Google would be qualified to work there, but since Google has it's pick of the litter they need to create a filtering system somehow. They happened to make it the "whiteboard" interview and similar high achieving Peer companies all tend to do the same thing. It became a thing, and as such all non-high achieving companies followed suit so they can behave like the high-achieving companies. The filter a company like Google applies is only necessary because they have too many candidates. Most companies aren't in a position to apply that filter if they really want talent. So really, they need to get lucky enough to get those who don't interview well with the "tech" company method of interviewing and hope they get some of those prime candidates.
I'm pretty sure the tech talent pool, as with all things in life, follows a Gaussian distribution. If you interview enough people you're bound to find some decent ones, even if they don't actually interview all that well.
I'm also frustrated with how people have developed disdain for anything whiteboarding. Asking leetcode puzzles in interviews is bad. Asking questions from CTCI is worse and amateurish. But... asking to whiteboard a problem that you had to solve on your own job in limited time is good! Interviewing is hard because crafting such questions in a way that it abstracts away minutia of transient technologies but retains essential problem solving is hard. This is exactly why, a right policy for a company is to establish being an interviewer as a privilege that needs to be earned because not even all brilliant programmers are good at it and interviewers must take time to craft good question and show care. When they don't, they turn to leetcode, pick random question and give whiteboarding a bad name.
And no, whiteboarding is not bad. What is bad is asking to do X using MongoDB and RectJS because technologies like that are transient. Specific technical tasks that employees might perform in a given point in time are transient. Ultimately what matters is ability to learn, process and use that to solve a given problem. Ofcourse, unless you are company who does nothing but data driven web forms. In that case, yes, go ahead and ask candidate to do just that.
As the article mentions though, the term is a bit vague and is overly broad; for example if you're doing a system design question, I'd argue that the whiteboard is the best choice for an interview, since that's how you're most likely to collaboratively sketch the design of a system in the real world.
I think the problem can be restated at a more general level; does the interview test the sort of skills that the job requires? Most people here are correctly pointing out that the skill/knowledge required to write out pseudocode for quicksort is almost entirely uncorrelated to the skills involved in most software jobs.
Making engineers write code on whiteboard is just comical. If we weren't so used to it, we'd probably have laughed at the idea. In 2018, any company still using whiteboards to write anything more than pseudocode is just playing it too safe or being really lazy in changing their process. Just give them a laptop and have them use coderpad/skype interviews or any of the plethora of collaborative code editors out there. Or even, dare I say, an IDE. You can still have the whiteboard in the room to discuss the idea. But write the code on a laptop with internet access.
If the person says "we built real-time tracking for our in-house vehicle fleet", I will just probe further and further about what role they played and how they went about the specific implementation details.
If there is any kind of secrecy surrounding their work, I will simply offer them insight into what they'll be tasked with during their first week and ask questions in a similar manner.
I'm not sure what advantage any other approaches have - I am hiring to fill a particular position. It's quite likely that the candidate will be in a similar position, so either I ask them about their current work and how they went about it, how they intend to accomplish the work assigned to them when they work with me.
I have some firmly held opinions [1] on this topic. Interviews are too much of an "art" for how important it is that their results be repeatable. Keep them on topic as much as possible. Making a coder perform circus acts on a whiteboard they will never be asked to do otherwise is simply measuring for skills you don't need.
1. https://blog.benroux.me/whiteboard-coding-measures-penmanshi...
We have a round of interviews and then send the candidate home with a choice of 5 coding tasks. Intention is to spend no more than 2 hours. Then they return and hold a code review in front of the team and explain what they've written. That way, they code at their own pace and in a comfortable space where they can focus, much like the way they would actually work. Then they've got it all right in front of them for discussion.
The review then weeds out people who don't really understand what they've written, possibly because they borrowed it from somewhere or asked a shill to write it for them.
We've had a range of submissions, from the casual-careless-broken code to extremely complete to elegance like we've never seen before. It works well.
If the problem is not one of the standard ones (that will be perfectly memorized by desperate candidates, while rightfully confident ones may struggle), it will demonstrate understanding, not the irrelevant ability to sequentially write out code in an unfamiliar medium. After all, you probably don't want to hire those who have already been rejected so often that they can code on a whiteboard as if they did it all day.
The interviewer propmpted me along the way, it felt like a friendly conversation. The fact that it was an easy problem prevented me from stressing out, but I still felt like I was able to demonstrate some academic ideas.
I was always pretty decent at standardized tests as a kid, and this argument over whiteboarding reminds me a bit of that. I don't think it's useful for gauging general software engineering skills, but it might have some value for evaluating communication. It also feels like the sort of thing that software engineers with a strong academic background tend to enjoy more than those with more practical/industry experience, but that's just anecdotal.
Does this practice exist anywhere?
It would be interesting to do this maybe once a year? And put $100-500 toward getting a pass or non-pass(money gets donated to interviewer-selected charity), just to throw in some money in the game?
And I mean an interview based on non-trivial CS problems, not brainstorming / code ideation which we do often.
Whiteboard coding does a good job at assessing that, regardless of the candidates ability to actually solve the problem (for all the reasons why whiteboard coding sucks).
Do you analyze the problem?
Write test cases first?
Write a mini spec?
Gather Requirements?
Ask Questions?
Jump into code?
Google?
Verbalize the problems you encounter?
In that respect, engineering coding interviews work well. But too often, they are essentially a binary Yes/No: Candidate solved the problem the way I want them to solve it.
Whiteboard as writing programming pseudo code or steps are fine. Just don't be a human compiler and demand perfect syntax.
If the test is to see how well a person works in a whiteboarding interview to solve a problem, then it should be a simulation of collaboration, while trying to avoid the Clever Hans effect.
Otherwise your DP whiteboarding problems are worthless.
Instead, how about sitting down in front of a computer with the candidate, going to an interesting problem on the LeetCode site, and reading the problem statement with the candidate. Make sure you both agree on what the problem is asking for.
Then, instead of having the candidate solve the problem, go to the discussion page for the problem, and start reading the posts with the candidate. These will include posts from people who are asking for help understanding the question, and from people who are showing off their solutions.
Pick a few where the poster is asking for help understanding, and ask the candidate how they would clarify things for the confused poster.
Pick a few where the poster is posting their solution, and ask the candidate to do in informal oral code review along with you. Let the candidate take the lead, just providing nudges if necessary. For ones where the review finds serious bugs, talk about how to make test cases to demonstrate the bugs.
From what I've seen on LeetCode problem discussions there will be a lot of posted solutions that have edge cases that fail, or fail to meet time or space requirements that were given, or leave out major cases, or make implicit assumptions about the input that if violated invalidate their sultion. There should be no problem finding plenty of posts with plenty of things to bring up in review to discuss with the candidate.
This seems to me that it would not be overly stressful on the candidate (way less stressful than writing code under a short deadline), it is exercising skills that working programmers actually use, and it would let you do some work with the candidate to get some idea of how they get along with others when working.
But instead of acing it an issue cropped up in the coding pad which I had to try and debug, which threw me off guard. The interviewers had muted their mic and were chatting away and my nerves just crumbled and I ended up profusely sweating into my eyes, I literally had sweat going into my eyeballs. I just turned into a bit of a mess, which is strange for me as I'm normally quite confident.
I think if the interviewers were a bit more involved it may have gone better. But I ended up just not finishing in time feeling like I had wasted everyone's time.
If the candidate can't explain or solve something simply on a whiteboard, they probably don't understand the problem or domain.
That's not to say that the interviewer might not be a great interviewer in the first place though.
Quote: "Feynman was a truly great teacher. He prided himself on being able to devise ways to explain even the most profound ideas to beginning students. Once, I said to him, “Dick, explain to me, so that I can understand it, why spin one-half particles obey Fermi-Dirac statistics.” Sizing up his audience perfectly, Feynman said, “I’ll prepare a freshman lecture on it.” But he came back a few days later to say, “I couldn’t do it. I couldn’t reduce it to the freshman level. That means we don’t really understand it.”"
Instead, have a conversation about how to solve a well defined problem. Use the whiteboard to record potential solutions in high level terms, or if the candidate chooses, to note key concepts or concerns.
Your interview environment should be a "staging" environment for your day to day operation. The interview should gauge how the candidate might affect your development process.
Once the interview ends, snap a picture of it. When you're debating candidates and try to remember your impression of them and how they contributed. Consider the sum of the input they verbalized and the input they wrote. That should give you a good idea of what they'll offer - well, along with your impression of them to that point.
Of course I have no problem testing architecture and communication ability in front of a whiteboard, as this is much closer to how these things are done in the workplace.
I wrote a book it - along with what I've been using the last few years, that actually seems to work - or work at least a bit better. In the end, I don't think you really know unless you work with someone.
https://www.amazon.com/dp/B07FJ6N8P1
It's my first attempt at a book (small as it may be!) - I'd really appreciate some feedback (and it's free right now)
-Greg
In practice we do end up at a board day to day explaining ideas and so on, and everyone seems more than comfortable (with the exception of handwriting insecurity).
Less experienced engineers are very difficult to assess without some kind of coding exercise. All the hires I've made that I'd characterize as mistakes are of junior engineers that I didn't do a coding exercise with and interviewed well, but didn't bring that interview polish to the job. Since instituting whiteboard exercises for junior positions, our success rate has been higher (no pun intended).
Silicon Valley is retarded.
Spitballing solutions to a problem collaboratively and with access to reference resources (a nice little programmer's bookshelf in the room) followed by actually coding a little bit up seems that it would be way less stressful and way more likely to give you a sense as to what someone is capable of and what they're like to work with.
It's not sufficient to decide if someone can code. You want some actual code written by that person.
And you don't need everyone on your team being able to explain things well on a whiteboard. Just try to figure out how they are thinking and how they are approaching a problem.
Recently I interviewed at Google, and they had no problem fulfilling my request to do both: sketch the solution approach on whiteboard and then write the code with a laptop.
but it very much does seem like whiteboarding is on its way out.
Is the typical engineer conducting an interview good at contextual inquiry? At holding space for people to think out loud? Lets not kid ourselves here. The only people we’re harming are each other.
And yet, whiteboard interviews are probably 3rd on the list of flamewar topics behind vim, and whitespacing. So there is clearly some people who don't hate them.
I count myself in the camp of people who don't relish the world before fizzbuzz.
developing under such constraints is radically different from environments with high iteration velocity - i.e. browser or scripting.
that said, in many modern environments whiteboarding is completely irrelevant, as modern application level programmer productivity is often ability to do plumbing-and wiring- work between many components and quickly assess why something is going wrong, rather then creating a highly performant or memory-efficient algorithm implementation.
https://medium.freecodecamp.org/how-to-organize-your-thought...
Just embrace it. It will be okay.
If yes, the problem should be simple and relevant, and the session highly interactive. The assessment should be about (a) How interested the candidate is in the problem (b) How receptive they are to feedback.
I still only do OK at whiteboarding in interviews.
A whiteboard interview isn't a good growth indicator, but it's great at leveling a candidate. Moreover, it signals to a candidate that their peers will be at a similar level.
So I'd vote yay. If you pay attention, you can determine someone's proficiency at coding on the whiteboard in about 15 minutes. This is particularly important when you have people who are changing careers or junior or otherwise don't fit the mold.
I'm honestly shocked at the arguments against whiteboarding as many of them seem entirely reactionary.
"But I'm never going to work that way."
Didn't we all do the better part of two decades of formal schooling? We test people like that for a reason: you can carefully focus on the specific markers of mastery of a subject.
I understand why people make this argument: it's a reaction to some of the weird puzzles companies used to ask. But it's never substantiated that this will help the interviewer and candidate make a better decision, so it has the same problem the puzzles did.
"But at work I have Google and StackOverflow."
This is a good point, the questions you ask in a whiteboard question should be based on the fundamentals rather than trivia. But it's a terrible point because it accepts that the interview is just a ritual you go through and this argument is that we should make it fair.
Whether a candidate knows "the answer" is not the point, it's more that they're able to talk about what's going on, have good instincts, etc. If you're an experienced engineer, you've seen lots of people write code, you know what looks like "junior" or "senior" and "guy who thinks anyone can just pick up coding."
"But if I work on a computer I can compile and see the edge cases."
Are you interviewing the candidate or the compiler? Again, this is the ritualistic view of interviewing.
"But at work I have plenty of time to work on things."
Everyone has this problem, so everyone is less able to perform, and the interviewer can simply adjust expectations.
"All of these interviewing "tools" (or tricks) attempt to be time/cost efficient proxies for doing the damn job..."
That's very much the ritualistic view. Other industries have had notions of apprenticeships and so forth for thousands of years, and our problem is that we view recruiting, professional development, etc. as things someone else does, rather than something we take ownership of. Interviewing is a critical business function, it can produce value and thus is not a waste of time. If you treat it as a useless ritual, though, it will be.
Lock me in a dark closet under the stairs with just the glow from a couple LCDs and throw pizza and requirements documents through the slot, like you're feeding the Rancor. /s
"Highways: Yay or nay?"
"Food: Yay or nay?"
"Air: Yay or nay?"
Hate on whiteboard interviews all you want, but that attitude is going to put you a massive disadvantage in the current rigmarole of software engineering interviews. It's not a thing 99% have any interviewees have control over. Nor is there a way to tell the interviewer: Hey, I prefer not to do whiteboard interviews. It doesn't work like that.