Please stop the coding challenges (opens in new tab)

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

All the time. 300-400k SLOC in C++. Legacy in the sense that there were no tests of any kind. Little-to-no documentation. Solo developer at the tiny company. Fix bugs and add features while keeping the system available to the tens of thousands of users.

A more recent example: here’s a patch for a critical feature we need. It was written over a year ago. The original author isn’t available anymore. You can write the code from scratch or try to resurrect the patch against master.

Being able to jump into a project and lead people towards some goal is definitely a skill for senior developer positions. Yes, you generally have a team you can lean on and have the ability to do research and all that. But how do you show that you can do all that in an interview?

Agree with the conclusion that a good thing to test for is for problem-solving.

The tech side depends a lot on what you’re doing. Although it gets ridiculous and organizations get lazy with this part. You don’t need to be white boarding graph algorithms for a junior web developer role. If your application is a social networking role and you’re interviewing a senior developer or architect? Definitely. They’re going to be teaching this stuff and need to understand it at a deep level.

A small anecdote.

A partner of a friend quit their job earlier this year. They then took 4-6 weeks to prepare for each interview with Big Tech companies (4-6 weeks for Meta, 4-6 weeks for Stripe, etc.). Along the way, they also took random interviews just to practice and build muscle memory. They would grind leetcode several hours a day after researching which questions were likely to be encountered at each Big Tech.

This paid off and they accepted an offer for L6/staff at a MAANG.

Talked to them this week (haven't even started the new role) and they've already forgotten the details of most of what was practiced. They said that the hardest part was studying for the system design portion because they did not have experience with system design...but now made staff eng. at a MAANG. IRL, this individual is a good but not exceptional engineer having worked with them on a small project.

Wild; absolutely wild and I feel like explains a lot of the boom and bust hiring cycles. When I watch some of the system design interview prep videos, it's just a script. You'll go into the call and all you need to do is largely follow the script. It doesn't matter if you've actually designed similar or more complex systems; the point of the system design interview is apparently "do you know the script"?

Watch these two back to back at 2x speed and marvel at how much of this is executed like a script:

- https://www.youtube.com/watch?v=4_qu1F9BXow

- https://www.youtube.com/watch?v=_K-eupuDVEc

I recently ran an interview process for a relatively senior eng role at a tiny startup. Because I believe different interview methods work better for different people, I offered everyone a choice:

1. Do a takehome test, targeted to take about 4 hours but with no actual time limit. This was a non-algorithmic project that was just a stripped-down version of what I'd spent the last month on in actual work.

2. Do an onsite pairing exercise in 2 hours. This would be a version of #1, but more of "see how far we get in 2 hours."

3. Submit a code sample of pre-existing work.

Based on the ire I've seen takehome tests get, I figured we'd get a good spread between all three, but amazingly, ~90-95% of candidates chose the takehome test. That matches my preference as a candidate as well.

I don't know if this generalizes beyond this company/role, but it was an interesting datapoint - I was very surprised to find that most people preferred it!

The more people online complain about coding interviews, the more confident I am that they are the absolute best way to filter candidates for a software development job. Across the industry there are way too many talkers/pretenders/meeting schedulers and not enough people who can roll up their sleeves, jump into the code and actually get stuff done. And this problem becomes worse at higher levels. You can bitch about it all you want, but you aren't owed that cushy $500K/yr FAANG job. If you can't get yourself to brush up on basic programming and write some for loops then companies will simply move on to someone who will.

> This is like asking a Ruby developer to debug PHP as a test of flexibility.

Sounds like an OK test to me. Great (senior) developers should be able to do that kind of thing. Categorizing yourself exclusively as "a Ruby developer" is a career trap.

> These high-stress, solo coding assignments don’t reflect the actual job. Instead, they put developers in situations they’d never face in the workplace, where collaboration and support are standard. When was the last time you had to debug an ancient codebase without documentation or help from a team?

I have had to do this, on numerous occasions. Sometimes there is a system that no one who built it is left, the documentation doesn't exist or can't be found, and that's it. This is not a great example. Being able to debug a program, follow requests around and see what's happening is a good skill.

What this example is really bad at though is that you pigeonhole yourself into developers. If you have a legacy php application, you are weeding out the java, python, ruby, go etc developers that do have those skills but don't know php and will be slower at it. Despite them possibly being stronger developers given a couple months of hands on php.

> A “4-hour” assignment can quickly turn into 8, 10, or even more, just to ensure it’s in great shape.

This I agree with. Suggested time can easily be gamed, so someone who has more time can look like a better candidate.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Literally every day this week.

Documentation? The code was last updated in June, the documentation was last updated in January - of 2022.

The team? The half that didn't get laid off two years ago all quit for greener pastures. Sure, there's a team that owns this project - along with three others, all of which they've contributed maybe a dozen lines to over the past year, and 11 of those were dependency updates.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

> This is like asking a Ruby developer to debug PHP as a test of flexibility

> If the job requires specific tech skills, test those skills

Sounds like someone failed a coding challenge.

In all seriousness, you need to understand that companies rarely implement their hiring practices for the sake of it. It is usually the best their collective minds could come up with. Telling them to get rid of it without attempting to understand what problem they're trying to solve and without offering alternatives, is, to say the least, not a productive use of anyone's time.

I find in software development we try to reinvent the wheel all too often instead of borrowing practices from other professions. Let's have a look at what civil engineers have to go through to get hired at half the salary of a software dev, and let's incorporate that into our practice. That will make everyone a happy trooper !

I had a very bad experience at a company that did not conduct any coding tests; there were just 2 or 3 interviews, and that was it.

The issues began when we started working together in a Data Engineering team. Most of our stack was based on Hadoop, Kubernetes, Ruby, Python, and several other technologies that required a basic understanding of cloud computing.

However, since we had such a wide range of work experiences and backgrounds, I often found myself not doing the work I was hired for but instead covering for others. I ended up doing a lot of "glue work" to compensate for the fact that some colleagues couldn’t even handle basic tasks, such as Python CLI packaging using Docker or inspecting jobs in Hadoop. After some time, I decided to leave—not because I thought I was special, but because I was fed up with spending 80% of my time providing support for people who were hired at the same level or even higher than me.

I recognize that the company had very poor hiring practices, but some form of basic technical testing is necessary.

> A “4-hour” assignment can quickly turn into 8, 10, or even more, just to ensure it’s in great shape.

The fun part of take-home assignments is that there's a moral hazard incentive to spend extra amounts of time on an assignment even if they say "only spend X amount of hours on it," even lying amount the amount of time spent if the interviewing company asks.

More than once I have completed a take-home assignment with emphasis on sticking in the timeframe they recommend but still solving the problem adequately, and then I get chewed out in the followup interview "why didn't you implement X?"

Relatedly, a massively open-ended take-home assignment is a bad take-home assignment since it favors those who have more free time to be thorough. (shoutout to the take-home data science assignments which ask for the final deliverable to be a PowerPoint presentation)

Remember that in other fields like medicine, finance, academia, and law, getting in involves 5+ years of hoop jumping and commitment signaling that have nothing to do with the final job.

We are blessed.

I hear it all the time ‘coding interviews are useless’, ‘peer review in scientific journals is broken’ and so on and so on.

I’d say yes and no.

Yes, these are the problems that cannot be solved perfectly.

No, because in such areas any ‘reasonable’ filter is better than nothing. People say that these assignments don’t have anything with reality, but, well, we don’t have months to try to work with each other, we only have 3x1h.

I worked as individual contributor for years, but also had a chance to try a hiring manager role for the past 3 years a lot. We do standard leetcode-style interview (without hardcore) + system design. And I always consider both as a starter and bite to see how the candidate behaves; talks; do they ask questions to clarify something and how. And I always try to help if I see that candidate is stuck. By the end of all interviews you will have some signal, not a comprehensive personality profile. Do we do mistakes? I’m pretty sure, yes. But I think it just works statistically.

As someone who interviews i support these tests, my reasoning is very different

1. Applicants getting hired because the manager and the candidate is from the same institute or they like the same team or band.

2. Bias based on age or race. I try to remove my own sub conscious bias as best as I can but most don't.

3. HR thinking you are not good enough and rejecting because you did java 18 and not java 19. Or you can write impressive React but haven't done Vue.

It takes the managers and HR a little out of the equation. They just rubber stamp your decision.

Take home exercises are something we tried but the incidents of cheating were so high and good code is so subjective that we gave up.

Companies aren't incentivized to eliminate false negatives in hiring (talented people getting overlooked), they just need to make sure there are no false positives (bad hires). I'm guessing it's probably rare for companies to think about the bigger picture, like the long term health of the economy and nation, where it's actually really bad if there are a bunch of really talented people that are being underutilized.

If I were interviewer, I'd probably do a coding assignment that builds on an already existing mini codebase, quarter or half day assignment (2-4 hours), with a strict time limit to turn it in by the end of the allotted time. It would test for someone's ability to read unfamiliar codebases, with the codebase being reasonably small for the little time that's available, and the ability to build features on top of it, to write clean readable code compatible with the codebase, etc. And an a 30-60 minute more open-ended informal knowledge interview to gauge how knowledgeable the candidate is and where their strengths and weaknesses are. And previous work would probably be a good indicator, like open source work and/or previous job experience.

I think the analogy by Steve Yegge still holds: if you claim you're a juggler, you should be able to juggle in front of people at any time. So, yeah, coding challenges is a good filter, at least.

The problem is not coding challenge per se, but that people can now cram it on sites like leetcode. See, before we had leetcode, only two types of people could solve a large number of algorithmic problems organically: those who were naturally talented, and those who were so geeky that they devoured the works of Martin Gardner, Knuth, and the like. Excelling those challenges showed raw talent in old days.

And companies like the young Microsoft and young Google absolutely loved such talent, and such talent did shine in those young companies.

> build a mini-app from scratch in just a few hours

It depends on what kind of functionality we're talking about, but this kind of task is exactly what people at my current startup have been assigned at times. It is absolutely possible to build a CRUD web app with reactive UI using modern tools in a few hours.

> This is like asking a Ruby developer to debug PHP as a test of flexibility

Again it depends on what the debugging task is. At every startup I've worked at, it's expected that an engineer is able to jump into a task that they know very little about. Granted it becomes less reasonable the more niche the task, but PHP and Ruby are not particularly far apart in skillsets in the grand scheme of things. I would expect any web engineer to be able to do this.

> Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas

I agree with this. And, hiring should also focus on technical ability which does include working through difficult and unknown problems by oneself.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

I've had to do this at some level in every job.

Somebody left. The code was left to rot, whether it was a separate codebase or it was an entirely separate project. I got assigned to fix it. Sure, I had a team, but nobody knew more than I did, so it was just other devs who might have another perspective.

Collaboration and support might be standard, but mostly in the form of other smart people, not always documentation, up to date software, or people who know anything about the code.

I personally prefer these kinds of challenges as they mean I don't need to maintain two skillsets.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Yesterday! Working on a large project means there’s always some issue with a dark corner no one has looked at recently and because there’s no team to support, I get to go and bug hunt.

Code challenges are great if well made, and horrible if not. They have to be short enough not to be unpaid labor (few will go through the trouble of paying their victims to work on interview tasks, and if I have an existing job I don't want to plow even 8 hours into getting any new job).

But having a simple coding task does some great things in that it will make the interviewee have to fix problems. They need to ask when unsure. You'll know who stops and asks and who plows through and solves the wrong problem. You'll know who throws their hands up when instructions are unclear and just says "I can't do this" and gives up. It's one of those extremely valuable and very effective filters that you just shouldn't be without.

The actual assignment can be however simple, but if they manage to install the tools, clone your repo, read the instructions, build the thing etc. then you already filtered out half the applicants or more. And that's without even starting to code.

The take-home challenges at my last jobs have been close enough to the work to be useful, and much more relevant than anything else I've seen in any interview.

> Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas. Unrealistic expectations don’t attract the best talent – they just exhaust and discourage it. If companies want adaptable developers, they should focus on the long-term ability to learn, not how fast someone can tackle an arbitrary test. Dropping these absurd assignments and focusing on what really counts could foster a better, more inclusive tech culture.

Sure, how? There are no perfect processes, only drawbacks that you can live with.

I also find the idea that interviewing is somehow exploitative of the interviewee ("feel like working unpaid shifts for a job they don’t even have yet") to be somewhat bewildering. It's not like your coding challenge is useful work for company that you are doing for free, in most cases interviewing is also a huge time sink for the company as well. Not every interaction should be quid pro quo...

At most places I know, a coding challenge is not "the way" to get accepted, it is a way to get rejected.

The hiring manager might hd really liked a candidate and conclude that a coding challenge just shows that the new hire would need extra time to ramp up.

Yet, ignoring money, you could excel in the coding part and the hiring manager might not like you because you lack "X" (e.g. "communication skills", "team culture", etc), which in reality is a way to put they just didn't like you or there were not really looking for someone to deliver, example, they were looking for someone to play politics or that might benefit the hiring manager more in the long term.

Not always the more technically capable candidate gets the job.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

This is pretty much my day job. OP is spoiled if they think this is an uncommon situation.

Whiteboard coding interviews are underrated. Sure, they suck to learn and master, but once you gain the experience it's a passport to a wide range of top-shelf jobs.

Can people game them? Are they flawed? Absolutely. But that's true of literally any interview practice.

The idea behind whiteboard interviews is if they're challenging enough, the only people who can fully exploit them are either very bright, very motivated, or some combination of both. That's a better signal than asking someone to lie about their "passion," which any slob can do.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Pretty much all the time. I also found that debugging things without asking for too much help was a great way to learn the systems involved, which eventually made me a better and more useful programmer, with more domain knowledge.

Granted not everyone has the luxury to spend time doing this, but I don't think it's that rare. In many cases it's even what companies want you to do, when that documentation or help is simply unavailable.

I see lots of complains about coding challenges but what are the suggested alternatives?

Here the article suggests "Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas". Ok, but how? Problem solving is typically tested using puzzles of some kind, like... coding challenges, but apparently, it isn't the right way, so what is the right way? And how do you test collaboration before hiring? The hiring process is competitive by nature. As for growth, again, hard to test in advance.

Suggestions I have seen are:

- Interviews: great in theory, but some people are great at bullshitting, others are competent but just don't do well in interviews

- Open source contributions and past projects: great if you worked for companies doing open source before, not great if your past work is confidential

- Assignments: Hopefully paid, but in any case, very time consuming

- Puzzles, IQ tests, school-like knowledge tests, ...: like coding challenges, but worse

- Probation period: hiring someone just to fire him a week later is not great for morale

- Dice rolls: maybe worth considering

Coding challenges have a number of advantages. They are unbiased with regard to age, race, gender, religion, etc... They test a skill that is at least related to the job. They can be done remotely and don't have to take that long. Whatever replace coding challenges should meet these criteria too.

The part I always feel like is missing from the coding interview discussion is:

"Are you testing the candidate on what the job will actually require, and ONLY that?"

With an onsite whiteboarding challenge, you're most likely testing if they're really good at thinking and talking in front of a crowd when extremely nervous, not writing code.

Same with a live coding challenge over zoom (CoderPad style) - this tends to be a test of nerves more than coding ability. But, if you're planning on a lot of pair programming, and want them to be able to dive into that on day one, maybe it's the right way to interview.

I find take-homes often the most accurate way to interview, as it most closely reflects the reality of most coding jobs. You're on your own, you don't have to deal with the extremely odd (and for some of us terrifying) feeling of someone watching you type. Hopefully, the take home questions are carefully designed to reflect what the job requirements will be. As almost every other comment has pointed out, debugging an ancient code base without documentation may be a perfect question, or it may be completely irrelevant and unnecessarily eliminate candidates.

But please, make the interview match the requirements you're actually hiring for, otherwise, everyone's time is being wasted.

Some problems I see with take-home assignments:

* Some companies design take-home assignments that mirror challenges their team has spent years refining, setting unrealistic expectations for candidates to match or exceed that work. Talent is sometimes dismissed over minor differences, like testing styles or even coverage percent.

* Working with people is about compromise. When passionate people collaborate on a creative endeavor, there's always some back-and-forth of ideas. But in these assignments, often only the unilateral opinion of the evaluator matters. I guess being rejected from a monoculture early in the process may actually be a blessing in disguise.

* Many companies fail to provide specific, actionable feedback after candidates spend hours on assignments.

I don't mind take-home assignments, but I avoid investing excessive time in them, as it’s not sustainable and often has low ROI.

I have literally 10 years of experience in Android development (starting with Java and transitioning to Kotlin around seven years ago). There are many topics in Java and Kotlin that are irrelevant to Android development, but I focus only on what I need. I quickly learn and build on new topics when required.

I’m speaking here as a developer who has done more than just bug fixes or UI tweaks. I’ve built apps that have served millions of users. I even have personal projects that currently serve 20–30k users daily, with over 1 million downloads and a 4.9-star rating from 41k reviews. Beyond Android, I have experience managing servers with Nginx, setting up SSL certificates, using Cloudflare, AWS, and Hetzner.

I’ve designed an app with users, posts, and likes, handling both the front-end and back-end using cloud functions and Firebase. From writing Firebase rules to encrypting everything, I understood and implemented it all within a few months. I’ve also written numerous web scrapers in Python for personal projects, which are still in use. While some backend concepts might seem basic, I believe knowing how the backend works is a huge advantage for an Android developer.

Despite all this experience, I recently failed to solve a "number of islands" problem in an interview and got rejected:

Given an m x n 2D binary grid representing a map of '1's (land) and '0's (water), return the number of islands. This left me feeling useless. It made me question whether I’m fit to be a software developer. I studied DSA in college, but now I struggle to recall or prepare for it. Does this mean I’m a poor developer? Does someone who grinds LeetCode just necessarily have better skills in general? Is this what companies want? Are they okay to spend money and time in training someone just grinds LeetCode for the purpose interviews. Does someone who does this just for interviews is likely to stay in same company?

I believe there must be a better way to evaluate candidates. It makes sense to test DSA in campus interviews, where students are fresh and studying it. But for experienced developers, especially for roles like Android development, DSA is rarely relevant to the job requirements.

I’m not saying I can’t practice and do DSA—I can. But it has never been a job requirement for me, and I doubt it ever will be for an Android developer or a similar role. If it’s ever truly needed, I’m confident that most experienced developers, including myself, can excel through research and applied knowledge.

#Kill-DSA

i think we need to come to terms with the reality of coding challenges in the interview process. i know i hate them personally, and dread having to interview again because i'll need to open leet code and remember how to do stupid shit like DFS on a graph, or manipulating linked lists. at the same time, a job opening for a SWE is opening soon in our company and we'll have to somehow filter people, and the job market is such that we'll get MANY applicants, most of them probably wrong for the job. i will probably end up giving them coding challenges (not necessarily leetcode, but some coding challenge for sure), because i need a way to grasp their problem solving and coding skills. i don't know a better way to do it in a condensed time frame of a 1 hour zoom call.

Had the weirdest interview of my life on Tuesday, more on that at the end.

Relevant part: the interviewer/CEO's philosophy is not to code challenge because no one properly vetted for "problem-solving, collaboration, and growth in relevant areas" (tfa) would ever accept a job they were going to fail at. Will always stick with me.

Now, here's the weirdest interview of my life: An acquaintance who runs a software business interviewed me by bringing me along to an NBA game he has season tickets for. The interview in the dining area before the game was normal, the game included some breaks for more interview discussion, but the home team had the best shooting night of their entire season so the crowd was very rowdy and we each had 2 beers.

I apologise for the shameless plug but I’m working on something to solve this.[1] The problem is that most hiring can be classified into 2 types - the company needs candidates that can work with their tech stack OR they want candidates who are good, regardless of tech stack. What we do today is conduct interviews that make no sense and that have vague signals when we could be assessing them based on real-world skills. Candidates shouldn’t have to practice interviewing in any case!

If you are a hiring manager and would like to experiment with something like this, please get in touch!

[1] https://pencilbeam.com

I'm fine with tests, but only if companies pay a standard fee (say, $100) for a dev's time. If a company doesn't respect your time during the interview process, it probably won't while you're on the job.

I feel like the real problem is that most people are poor evaluators of talent. The pervasive bad interview processes in industry reflect this truth (as well as cowardice that allows people to hide behind process to avoid accountability as well as accusations of bias). The only way to escape this trap is to find the right connections or start your own org. Otherwise you will spend your entire career suffering the consequences of broken interview processes (both in terms of your experience getting hired as well as being forced to work with bad hires who game the system).

I once ended a job interview at a company because of their coding challenges at the end of a 2-3 hour interview. It was a job for a PHP developer working in a DDD environment. They were asking people to do algorithms, the last one just took the cake because it was completely unnecessary.

The question made sense: how do you add to very large numbers together? I assumed the key knowledge they wanted was that the numbers had to be represented as strings and you should use bc_math to compute them. So I answered that, and they said, "Yeah, great. Now write a function to do that." I sat there for a few moments thinking about it and realized it was just nuts. So I asked if I could just end the interview process. They were clearly shocked, I doubt many people end an interview process on the last question. I said that this wasn't realistic and that if I did that for real it would get code-reviewed to hell. One of them kept saying but it's to see how you would do it if bc_math wasn't installed. I simply stated "I would have them install it" to which one of the interviewers instinctively nodded since that's the real-world answer.

It seemed like it would have been a good place to work up to that point. I didn't like my current job at the time, it paid considerably more, had more public holidays due to weird German reasons, etc.

I have a very different perspective on coding challenges. In my last role building a startup, I WAS the employer. The results of coding challenges and the conversation that followed were the most telling about a candidates technical and reasoning skills. I would genuinely never hire without them.

If you're not willing to do coding challenges, your ability to write code and reason about your decisions probably suffers as well. It's self-selection.

We told candidates to not take more than 2 hours on our challenge and we meant it. I tested the challenges myself from scratch to verify that it was possible in 2 hours. I still only consider myself a quasi-engineer, so if I can get it done in that timeframe, a full-time engineer using AI should surely be able to.

There's a few considerations that I believe make coding challenges as fair as possible:

- Make sure the challenge 100% relates to a real task they would do on the job. As an example, for our engineers who would be writing open-source Python ETL integrations, our test was to build a Python ETL integration to pull episode data from a IMDB api.

- Tell the candidate that it's ok to not clean or refactor the code. Meet the requirements of the task first. Your job in the interview process is to ask them what could be improved or why specific decisions were made.

- Don't have anyone perform a technical test until you've first invested a decent amount of time into them as the employer.

- Ask the candidate how long it took them. If a candidate takes longer than the allotted time, they're showing you that they'll overwork themselves and not follow instructions.

> Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas.

Agreed. When I think about the best engineers I've worked with-- and the worst-- I have identified certain characteristics that I want to look for in a candidate, and certain characteristics that I must avoid.

In particular, there are certain people who just can't entertain any thought or design idea that they themselves did not have. This makes them very hard to collaborate with, except in the cases where what we need to do just happens to coincide with what they are thinking. Related, there are certain people who can only think about exactly one way to solve a problem, and there is not room in their brain to honestly consider any alternative approaches. These people tend to be really bad team players. A good engineer will understand that there's no one right way to do certain kinds of things, that everything comes with tradeoffs.

During interviews I present coding problems where there is more than one way to attack the problem. Whatever approach the candidate does, I propose an alternative approach and ask them to talk with me about the pros and cons of the alternative approach. I also ask for them to extend upon the alternative approach and make it better. For a significant fraction of candidates, they just can't entertain any other way of doing the thing, there just isn't room in their brain to consider alternative approaches.

Ability to really listen and give an honest consideration for other people's ideas is a vital part of the kind of team culture we need to succeed, so I look for that in interviews. Long ago I abandoned worrying about leetcode in interviews, and nobody in more senior management has made me stop yet, and this is at a big tech company.

I don't mind the coding challenges necessarily, but more how they're implemented. I find them to be far too broad relative to the day-to-day, and it only showcases how quickly someone can write unthoughtful code.

For example, it's not uncommon for a challenge in finance to be "build an exchange matching engine that takes orders, modifications and cancels from different threads in two hours." That's totally doable if you just fly through whatever comes to the top of your head, and maybe have an hour left over to test, debug and modify it. But that project (an exchange matching engine) is something that the CME has 10 full time programmers working all the time on. A more realistic 2 hour task is something like "look at edge cases for the order cancel type, and figure out a way to handle malformed inputs." Then you can actually write production quality code for them to judge.

> What companies often ignore is the extra time candidates invest beyond the “suggested time” for these tests.

This is a feature not a bug. Companies are testing if you can focus and complete a hard uncomfortable challenging task, because at your job you’re expected to do things you don’t want to do, but will be rewarded for doing.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

I wonder why the author thinks this is something unthinkable and unrealistic. Small teams with strict ownership and skilled engineers. If you join a company like that you will inevitably find yourself in that situation.

This happens when companies have too many applicants. It isn't the best way to evaluate candidates, but it is a cheap and easy way to do it and resumes aren't enough.

It's common for candidates who have an internal referral to skip all these steps, because the referral establishes a base line of credibility.

We recently went through a hiring process using a recruiter. Our assumption was that the people they vetted were "legit" and we didn't need to do any additional work to verify that they had the skills they said they had. If the recruiter wastes our time it's a surefire way for them to never get business again.

Anyway, in lieu of a coding interview, I opened up a couple of tickets (one a long term change request, another a real life problem we encountered just the day before). I provided them logs and answered questions and watched the interviewees move through both scenarios.

We capped the time at an hour. The focus was more on the interaction between them and myself—it actually used the "real stuff" they'd encounter.

I was amazed at the diversity of the responses and how differently people approached a problem.

> Dropping these absurd assignments and focusing on what really counts ...

These assignments are so much closer to a simulation of doing the actual job than the industry standard LC interview. The only thing closer is work trial which has the significant problem of requiring candidates to not currently have a job.

I've recently had a couple technical interviews where they were structured much more like a pair programming session. Those were a great experience on my end, and for the interview I expect it gives a much more realistic view of what it would be like to work with someone.

I was asked to code a sudoku solver from scratch during an hour long tech screen. I got maybe 60% there. A few days later they called me up and said they liked my work but wanted someone faster. At the time I was bummed, but a decade later I’m glad I didn’t get that job.

This blog post lists problems but offers no solutions.

Yes, complain all you want about what companies SHOULDN'T do. But then what do you feel is a better approach?

From my experience, you ask 4 software engineers what a proper interview should look like and you'll get 4 different answers.

I was ready for this to be another complaining post about having to write code during an interview, but found myself mostly agreeing with it.

Having to spend time to do some work before an interview to prep (like a presentation about your experience or something) seems reasonable, but a company should be able to test technical skills adequately in the 5+ hours they spend interviewing candidates.

I feel like the take home assignment trend is part of the backlash against doing coding problems live in interviews and I agree with the article that it's just worse. It takes more of your time and also opens things up to candidates cheating even more than live coding over zoom does.

Just stick with the live coding and ask good questions.

I find that take-home assignments are very one-sided form of an interview:

- The candidate needs to do lots of work while the company gets away with little to none. - The candidate never knows what exactly the other side will be looking for, so the only way is to put in more work. Like when the assignment didn't ask for tests, they might still be expecting them. - The candidate usually gets no feedback about the work he did. More importantly he usually doesn't get a chance to defend his solution. It's either great or garbage. - While the company can learn something about the candidate, the candidate will learn nothing about the company by doing the assignment.

I've turned down a number of jobs in the last few years because of unrealistic interview requirements.

One asked for a 20+ page personality assessment (perhaps completing that _was_ the test?). The other asked for an interesting technical challenge which I was happy to do -- for fun -- and then followed up with a request to write a basic web service which was... unexpectedly simplistic for the level of role I was applying for.

I pointed out a number of my public, solely maintained repos on Github illustrating exactly the same sort of thing they were looking for and turned down the followup assignment, but we decided it wasn't a good mutual fit.

I’m a product manager, not a developer, although I used to be a developer a long time ago in a language far far away.

A few months before my last interview I had taken up Python by solving about fifty projecteuler.net problems. I’m not fluent, but capable, and since no one is hiring me to write code anyway, I put it on my resume.

At my interview with a CTO he said, I see you have Python on your resume? “Yep,” I answered, and said pretty much the above.

“Do you mind if I open a shared session to code in”

“Sure” (I haven’t coded in front of someone…ever?)

“Do you know what Fibonacci numbers are?”

“Yes”

“Can you write me a function to return the Nth Fibonacci?”

That’s clearly not a very hard task, and fortunately I was up to the task and got the job.

Frame shift: coding challenges are work. (I don’t care that the assignment is useless for the company, that’s not my problem!) The issue for me is that it is unpaid.

If you really want me as a developer, pay me. As an industry, we might even set a standard rate for these homework assignments. I suggest a rate which is enough for me to say it wasn’t a waste of my time.

As an anecdote, I was interviewing at a startup that sent me one of these homework assignments. I told them I would do it for $1000 now, or whenever I got around to it otherwise. I never got around to it.

FB interviewer had "two hard Leetcode problems within 35min", where 25min was background and 1st leetcode problem was "changed" (adding ambiguity to the point of be without solution - later could not find any full solution, for which I stated factors and disclaimers to interviewer).

Agree, such or similar unrealistic interviews are loss for both sides, plus they are plain STUPID. I've performed ~400 interviews that most included coding in my career with consistency and standardized report (can compare apple to apple).

Non paid solo-coding assessments should disappear. Many disrespectful companies send it automatically to all applicants to don't even look at them after.

Every time I get those I request to do a live coding assessment, if they are not willing to lose even 1h of their time they were not serious about you.

If you decide your process really requires a solo assignment you can offer a gift card to prove you appreciate the candidate (and don't be stingy, you are already saving a lot of money by not having 1 of your engineers interview)

No, I don't think I will. Thanks for the suggestion though.

> > When was the last time you had to debug an ancient codebase without documentation or help from a team?

I would rather do this than some tired ass LeetCode problem.

I had one coding challenge a month ago that said: this should take you no longer than 2 hours. It didn't have guard rails on it so you could technically spend more time on it. It ended up taking me 2 days on and off because it was basically build a front and and a backend. And then later I realized the instructions for submission were totally broken too: DONT make a PR - later in the instructions: make a PR.

Tech jobs should be more like interviews instead, rather than making interviews more like tech jobs. I'd rather solve algorithmic puzzles all day.

> they put developers in situations they’d never face in the workplace, where collaboration and support are standard

If you are the one doing the supporting of your collaborators it's pretty much just like a coding challenge.

"your coworker needs to Inver binary tree but doesn't know how, can you help him? ... also the Internet just returns interview worthy crap when you try to search"

I refuse to do algorithmic and / or timed coding challenges during interviews and I'm glad to see such articles pop up more frequently. These days, I get a kick out of pointing out to recruiters and managers that both my LinkedIn and GitHub profiles state clearly that I won't accept such interviews and they need to stop wasting my time if this is part of their process.

Pretty common artifact that take-home challenges are being more widespread now that the labor pool is oversupplied with tech talent looking for jobs. Companies need new ways to filter for candidates that are willing to do whatever it takes and work the hardest. The easiest way to do so is give them a "three hour" take-home assignment and see if they are willing to do it.

one of the biggest problems with coding challenges for me is the conflict between providing a good solution versus making a strong impression.

do you show them a quick'n'dirty solution that ignores edge cases but shows i'm a pragmatic and not going to overcomplicate things?

OR

do you show something fancy that you'd never actually do in an real codebase that shows off my depth of knowledge and where my ceiling is?

I agreed with most of the things in the article. However, the debugging test is very reasonable provided they give you a heads up on the environment/language you're meant to debug. Depending on the language, setting up a debugger can be a test in it of itself.

Debugging (and setting up the debugger) is a core skill every developer should have.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Literally my first internship. Had to migrate a small system from C to (modern) C++ in the span of 6 months. Most files (and I'm going to say in the ballpark of 85%) had last been touched in 2003.

I started my intership in 2020.

If one does want to get into FANNG-like companies or whatever those acronyms are, I'd suggest a few practical tips.

- Use probability as your advantage. Or put it another way, perseverance works. Do recognize that interviewing is a hit-and-miss game. Say with your preparation your chance of getting into any of your top 10 companies is 30% -- a pretty low number just for us to steelman argument. Let's say you can try each companies three times a year (most companies have cool-down period per department instead of per company). Then in 5 years, you can try 30 times. The probability of failing all of them is 0.3^30 ~ 10^-16. That is, practically, zero. Of course, this assumes that each attempt is independent of each other, and we can maximize such assumption by keeping learning the lessons of failures. By the way, this is also a reason why one can see so many so-so engineers in a big companies.

- Do not cram all of the leetcode questions. That's insane. Remember the quote from Invincibles: "when everyone is a superhero, nobody will be"? It applies to cramming too. Instead, use the time to study fundamentals. Pick up Kleinberg's Algorithm Design, Levitin's Introduction to the Design and Analysis of Algorithms, or Skiena's The Algorithm Design Manual. Note these books are not that academic. They all focus on the process of designing an algorithm from constraints and basic principles.

- Do not worry about Leetcode Hard. Yes they will be asked from time to time by an interviewer, but see my suggestion #1. If you really want to try Leetcode Hard, at least ignore the ones that require deep ad-hoc analysis or the ones that require an aha-moment. They are talent filters or memory filters or obedience filters, but they won't improve your engineering expertise. Then, why bother? And trust me, it is actually rare for you to get a Leetcode Hard question.

Whatever happened to looking at a resume and realizing this person can code?

What are references for anyway...

Using coding tests to hire developers is like asking a CPA to complete an arithmetic quiz.

They're just encrypted IQ tests, working around the Duke Power supreme court ban on IQ testing employees.

You're being tested on your general intelligence, not because anybody cares about binary trees or reversing linked lists or whatever.

This will only stop when we collectively refuse to participate in such theather plays, until then, welcome to what is ultimately a programming casting show with internal audience seating on their theather chairs for the audition.

I wholeheartedly disagree with this take. there's nuance obviously and takehome coding challenges can be done horribly but when done right and assessed properly, they are a wonderful tool for hiring.

People who say not to do something as an interview stage should propose an alternative.

Otherwise it's "well yes it's not great but it's better that not doing it"

Applicants underestimate the cost of a bad hire.

> "Hiring processes should focus on problem-solving, collaboration, and growth"

How is a coding challenge not problem solving?

Collaboration can be part of coding challenges.

How in the heck do you plan to measure growth in a hiring process?

>When was the last time you had to debug an ancient codebase without documentation or help from a team?

Uh.. just about every company i've worked at in the last 20 years has had little to no documentation and the guy who last touched the code left already.

The beatings will continue until morale (or the job market) improves.

In all seriousness, we just need to get together and refuse to do these types of problems (starting with hiring managers and strong candidates).

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

All the time now that WFH is normal and collaborating with a team member is 10x harder and 10x slower.

A recent related post with proposed solutions: https://news.ycombinator.com/item?id=41948474

Sounds like a skill issue

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

This has literally been my job for a year. That is an actually important skill to have.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Lol, I'm pretty sure that's been my job description for the past 20 years.

Take home test under 10 hours > getting stabbed in the eye by a blunt fork > live coding challenge for 5 minutes

we administer a coding challenge designed to ensure people can do some basic coding - we don't even have them compile it. But we have caught people who start talking about using a B tree to solve fizzbuzz and we know they're not a fit. It's useful as an initial screener.

>please stop the coding challenges sounds like someone didn't pass through coding challenge

There's a lot of talk from engineers about how this process is broken, but never a solution.

What gives?

My employer's coding challenge uses replit and takes 30 minutes in which we're watching them do it (and interacting, asking them questions or prompting them with hints). It's not a hard challenge at all--one candidate who'd been grinding leetcode did it twice in the 30 minutes, with different solutions. We do the same with a system design question.

Most finish it, some don't and still get hired. It's a workalong in which we see how they approach the problem, how they discuss it with me, and what they do if they get stuck. We tell them to go ahead and google things, just tell us what they're looking for.

Over 10 years, we've got a pretty good record of hires that are smart, professional, and effective. And without specifically trying to hire for diversity, we've had a surprisingly diverse range of hires. We don't care about "culture fit", it's more like "work fit".

So I agree that the sort of code challenge where you send them away and see what they come up with is both unfair and a poor indicator. The real hiring test in an interview should be "can I work alongside this person?"

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Is this a joke?

No leetcode.

No coding challenges.

No take-home assignments.

What is the alternative?

I did hundreds of interviews for a former employer (10k+ employees) and developed a fondness for in-person coding interviews. In particular I would tell the candidate up front that the goal of the exercise is to see how they collaborate on a problem, how they communicate about their thinking, etc. The problem itself was considered too easy by many of my colleagues, but I still found it an extremely useful for the seniority and capability of engineers.

A later employer was in the hiring tech space, enabling this kind of take home exercises. Lots of time was invested into getting signals as the developer was doing it because everyone knew that's just as valuable, if not more, than the final solution. But many companies using these take-home exercises do it just to filter out candidates. To them, it's a proof-of-work system that helps regulate the intake of their hiring funnels.

Eh, coding challenges in of themselves are fair enough. What companies really need to understand is that:

1. Said challenges should reflect the work done on a day to day basis, not something completely disconnected from the company as a whole

Google's interview challenges should be very different from say, a local web development agency's interview challenges. I've seen too many companies where either the challenge was way too difficult/complex for the job on offer (build a complex modern app from scratch for a junior/graduate level role, or solve a bunch of leetcode tests and HackerRank puzzles for a WordPress agency job), or where it was far too simple (I remember at least one software engineering interview for a React focused role where the challenge could be solved without writing a single line of JavaScript).

2. The requirements and environment should match a realistic working day

When was the last time you had no access to the internet, no access to an offline IDE, no access to premade software, no one to help, a boss standing screaming at you over your shoulder and a camera tracking your every eye movement in case you're 'cheating'?

Probably never right? Unless your company environment is a dystopian hellhole, this is probably not your typical set of working conditions. Don't expect interviewees to work under them, they're not kids in school doing an exam.

3. And that said projects should be somewhat realistic in terms of scope

Most people aren't coding a new website/app from the ground up every day/week. Expecting someone to create that in a few hours feels incredibly unrealistic, and an incredibly poor method of judging someone's skills in the role.

If you can't stand the heat then stay out of the kitchen.

This is a friendly reminder that you are welcome to withdraw a job application as soon as you are offered a coding challenge. If this is an exchange with a recruiter over email or linkedin messages, you can simply reply with something like this:

"Thank you for sending that along. I would like to withdraw my application for this position."

We tell candidates they will likely never have to do work that is similar to what we give them in coding challenges.

Do it anyway.

And if they don’t want to do it, we don’t want them.

While we're at it: enough with the brain teasers and weird analogies. These, too, have nothing to do with actual day-to-day work and are really just a way to exercise power over candidates.

> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Right the fuck now. That's what happen when you get saddled with whatever monstrosity the last coder(s) who quit commited or had to maintain themselves.

And then you get the joys of trying to frankenstein some modern js "component" developed like only React exists on some vanilla / jquery front. Also a personal rant for those modern js script kiddies: forms action attributes exist and work.

Aside from all the comments saying that coding challenges aren’t as unrealistic as you think, there’s also the fact that there seems to be an oversupply of good enough coders for some roles so companies need to filter beyond that. Yeah inclusive environments etc is a nice to have but if you can get a top level coder that can solve problems quickly and not crumble under time pressure, even better. Why settle for good enough if you can get really good instead?

As interviewees, we need to stop entertaining this nonsense. A company tried to give me a "homework" assignment and I just told them I'm not doing it and moved on.

Yet another rant like "stop doing what you do" but not actually suggesting how to test "actually required skills" in practice.

Nope. I gave dozens of coding challenges at my last job. It was as directly job relevant, and as low-stress, as we could make it. We were a Python+Flask shop, and we had candidates who claimed both those skills flesh out some API endpoints in a stripped-down sample repo. They could use their own computer, and their own editor, and collaborate with me, their potential new coworker. And I enjoyed giving the challenge, and liked making the candidate feel comfortable and excited to work on some of our code. We gave instructions ahead of time like "you're going to clone a Git repo, edit the Python, run the tests until they pass, then commit the results". You know, the absolute bare minimum you'd need to be successful at the job.

Some candidates with many years of listed experience had never used Git. That was a little surprising but maybe they'd worked at a Perforce shop or something. Nevertheless, they knew in advance they'd be using it today, and that's not exactly some obscure job skill we were asking them to learn and then forgot. OK there: "I didn't use Git at my last job but I read up on it, and I might have some questions." Not OK: "What's a clone?"

Others didn't have an editor or IDE set up. Just how? I get not taking work home with you, but you've never once written a little program to balance your checkbook or make anagrams or something else random? And again, we'd told them ahead of time they'd need it today.

It's OK not to be a Python expert, but if we give you:

  def add(a, b):
      return 4

and ask you to implement it, and you say you have more than zero days of experience, I do expect you to figure it out.

And I'll say it: we had plenty of people fail at the fizzbuzz step, often in astonishing ways. One memorable person couldn't grok the concept of factoring it out into a function like:

  for i in range(1, 101):
      print(fizzbuzz(i))

so that you could arbitrarily test fizzbuzz(1_000_000_000_000). They had the novel idea of capturing stdout and comparing it to a test fixture stringwise. Which, OK, it would work, but... I asked them how they'd see if the a test value in the trillions was successful and they mentioned using CLI tools like tail to check just the end. I did verify that they weren't just playing around for conversation's sake, like coming up with an absurd idea and exploring it for the fun of thinking through the problem. They were convinced that was the right way to write real unit tests in large codebases.

So no, I'm not giving up coding challenges, not until I see a higher passing rate among people who list years of experience on their resume. If you claim you've been writing Node apps for the last 10 years, by gosh, I want to see that you can at least write a working function in JavaScript.

I’ll act as a counter-balance on this topic. I have conducted hundreds of technical interviews across varying levels of experience - from interns and juniors all the way to PhD grads and those with lots of years under their belt. I have extensive hands-on experience in multiple fields and passable knowledge in others. If a candidate has worked on something in the past, a good position for me to be in is to know more about what they’ve done than they do. A passable position is one where they can walk me through what they’ve done and I pick up as I go along, piecing it together in my head with what I already know. I very rarely have pre-canned questions, instead I opt to come up with examples on the fly based on how the conversation (and it is a conversation) is unfolding. If the candidate is gold, I will usually have a first offer prepared for the candidate before the interview is over, and I do not cheap out on the offers. Candidates have historically reached out to express they enjoyed the interview even if an offer was not presented. I believe in respecting the candidate, respecting that people can be nervous and shy one day and social the next, that people interview differently, and that more interview rounds does not necessarily yield a better result. Prerequisite for this is you have to enjoy talking to people, put your bullshit ego aside, show your hand where you think it will help the candidate, and know what you’re talking about and what you don’t know. People can learn, people can adapt. Every so often, I accept candidates that would otherwise be rejected on the understanding that no “system” is ever perfect, certainly not one that involves people. It saddens me to see coding challenge, especially automated ones, because it shows laziness on part of the interviewer. However, it is a skill that must be attained and practiced like any other. And no, I haven’t used the return key since the 8th grade. AMA.

>please stop the coding challenges sounds like someone didn't pass coding challenge

So we should stop the coding challenges, stop LeetCode, stop whiteboarding, stop profiling candidates, stop asking for their GitHub page. How is hiring supposed to work then? Just post the contract online and the first one to mail it back gets the job?

I like live coding challenges, something like a ~2 hour pair programming session, ideally modifying an existing project. I invest as much time as each candidate, while we are both exploring whether we want to work together.