1 - Too much chatter. Part of the assignment is using judgment and working in ambiguity. I probably would have ran with what was given enough to knock out something small and local in an evening or two. Asking questions is usually fine, they even welcomed it, but seems counter to the original ask.
2 - Writing and sharing a proposal seemed like way overkill. You have to remember that these companies are now getting hundreds if not thousands if not tens of thousands of applicants, that is a lot to deal with if everyone does so. I think it's a bit of a disconnect...you feel like you're going above and beyond and being thorough, they feel like it's being a bit long winded and wasting time. That probably explains the nonresponse.
3 - The finished product seemed functional, but seemed a bit overkill on the infra and polish. This is probably a good thing to work with you, but ended up wasting a lot of your time if not being selected, which was the case.
4 - Maybe I missed something, but the requirement asked for terminal inspired. I'm not quite sure precisely what they meant by that, but didn't see any possible interpretation of that in the result.
Anyways, hope you don't take it too negatively or personally - you obviously are a talented individual and moreso seem to really care about your work. Just wanted to play a little devil's advocate with a different perspective.
"Create a terminal inspired email client so we can do an alpha test with some customers" is a reasonable ask for an engineer at an early stage startup. Of course, there would be a bit more specification, but a lot of the details would still be up to the engineer. This applicant wants more certainty than they can get.
This is illustrated by the line: "I would like to know what kind of response I could expect from Kagi if I drive it to completion." This is not a great request to make. There's no way they can answer that question, because there is no certainty available. They're probably getting a few hundred or a few thousand more submissions to evaluate.
Nobody is getting a few hundred or few thousand submissions to evaluate. Nobody. If you are getting 1k applicants, at best 50 are asked to do a take-home and even then, not all at once.
If by some miracle 100 people did this to completion at the same time, there should be a notice to the effect that due to high volume, blah blah blah.
There's no good answer and asking a question like that shows real narcissism imo.
> Part of the assignment is using judgment and working in ambiguity.
If trying to clarify requirements is not what they wanted here, they may as well ask candidates to pick a number between 1 and 10 and reject anyone who guesses wrong.
> overkill on the infra and polish
I know its an opinion, but hard disagree on both. Take home tests are intended to show off your ability. Without specific guidance, how can anyone guess how much showing off is expected?
Polish is only bad if it stops you from delivering. Rejecting something that was delivered for being "too polished" feels like you are saying someone did too good of a job.
---
In my opinion tests with vague requirements like this are more likely to be a different way of rejecting people who are not a "culture fit".
So yes, this is a culture fit test as much (if not more) as a design and coding test. Some people who are great at design and coding would fail it, and it's how this filter is intended to work.
Work environments that "code first, review later" have been some of the most toxic in my career. It really sucks when you spend days building a feature only to find its not wanted. Which is why explaining the feature in English and getting approvals is the industry standard for shipping projects.
This candidate followed modern software practices that healthy workplaces follow.
(I'm also a hiring manager at a 1k+ engineer company).
Process is healthy at large companies, because they move slowly and communicating your changes often becomes far more important than the actual code that needs to be written. Kagi is a small company. It does not make sense to cargo-cult practices from a company a hundred times larger.
But from my experience, the two types look at each other like the other is insane. If you write up an ERD at a scrappy 6 person startup, everyone is going to think you waste time.
Conversely, if you join a larger team with established processes and begin flinging code at the wall unabated, people are going to think you're reckless and possibly inexperienced.
I've worked in places with very strong product management, where every single detail must be approved by a PM. I've also worked in places where engineer has a lot of autonomy - "John from FOO dept is spending too much time wrangling the daily data updates. Can you help him, here is his email? Our higher-ups allocated two weeks for that, but we can extend this if needed." - and from then on, you are in full control of the design and implementation, subject to your team's rules (because no one wants a system with bus factor of 1).
For me, I've found that the latter places are much nicer to work in. The interview question seem to focus on latter situation too.
Exactly. The position they are hiring for is not that of a coder; coding is just one of the skills the position requires, maybe not even the most important.
> (I'm also a hiring manager at a 1k+ engineer company).
lol, this makes me very old school but I suffered jobs where that was the outcome of months if not years of my work!
* Can be completed in 30 minutes by a skilled programmer
* Has clear evaluation criteria, both objective and subjective
* Has multiple approaches that require making different tradeoffs
And of course, only give it to some candidates where the result will be make-or-break.
As someone who took one of these broad take-home assignments my last time looking for a job, I failed a the assignment for a job I was overqualified for because I was told I wasn't able to divine what parts of the extremely broad assignment I would be graded on.
I doubt I will be in a position where I get a job that isn't a referral for the rest of my career, but it really turned me off of these kinds of assignments, both taking and giving them.
While writing my questions (and testing in my teammates), I found that "can be completed in 30 minutes by a skilled programmer" very often means "can be completed almost automatically by AI", and that AI will give explanations too, that interviewer could repeat during code review phase.
I love that the industry has become so poor at gathering requirements that devs are now effectively filtered for their ability to mind read.
This project tests your ability not only to code, but to deal with ambiguity and open-endedness
>Delivery date: Sunday EOD March 30,
>This is two business days later than the two-week mark since you sent me your first email.
With no accompanying personal reason for the delay, it's already a rejection in my books. The objective was to design an appropriate solution within the deadline.
The youtube video provided by the OP seems more "web-app", "click driven".
For contrast:
* The OP's submission: https://www.youtube.com/watch?v=yY1sVXMkP_o
* Someone interacting with aerc: https://youtu.be/kpAwwgnZUxg?t=308
* Someone interacting with mutt: https://youtu.be/C35NRp42bEQ
1. Spin up an EC2.
2. Install Himalaya.
3. Do some configuration.
Code quality and a sound engineering approach are there.
Personal touches aren’t. Though they look for them, absent domain knowledge they probably don’t want to see them.
Himalaya has documentation and a Github already.
No need to invent the wheel.
Won’t take an unreasonable amount of time.
And looks like a first iteration of a tool.
If this had been limited to 3 hours then the worst case is that the candidate would have lost 3 hours, but far more likely is that they would have come up with an entirely different proposal and/or solution that was appropriate for that timeline, and that extra information would have made it clearer what the company was looking for.
The other point I'd always encourage applicants to confirm is: are you looking for any answer, or are you looking for a good answer. Some take-home tests are purely about passing a test suite and how you manage it doesn't matter, some would prefer that you meet 80% of the requirements but write better code. I've seen applicants do the wrong thing on both sides of this.
Surely there must be other ways to have an idea on a candidate’s skills.
In our case, hiring people with data engineering skills, we just ask of them a simple ETL challenge (pull data from zip file, transform it, insert into any database). We leave ambiguities in there and there are some Easter eggs in the dataset (eg null values where you would not expect it, incorrectly formatted CSV) that we use to evaluate how well a candidate can perform.
We timebox it at 4 hours, but don’t give guidance in case they run out of time, that’s a good suggestion.
During a follow up call, we review the code together and we’ll ask them questions on how to improve the code (“what if the dataset doesn’t fit in memory”, etc), which is what the actual technical assessment is. At that point they should already feel somewhat comfortable with the problem domain, and you can assess their real skills.
At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions).
If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?
People should be paid for the time they spend in interviews.
You make that the law and this ambiguous hoop jumping bullshit goes away real fast. Companies will optimize for cost instead of dumping cost onto the prospective employees.
This is good for the economy because it forces companies to innovate and optimize the interview process and it saves hundreds hours of totally economically unproductive time on the part of candidates.
So I have to wonder, is this a junior position, or did too much hadoop rub off on me a decade and a half ago?
Because you can't guarantee all candidates are spending the same amount of time, it becomes a game theory problem where the candidates will typically lose in some form. In many cases, the right answer is to spend extra time making a really polished (but not too polished!) solution and pretend like you stayed in the time limit. And every candidate is either a) doing that, or at least b) worried that their competition is doing that.
Even if we ignore that dynamic, 3 hours is a long ass time for a candidate to spend when they're not even sure they'll get to talk to another human about it.
In a 1-hour interview, you can run a candidate through a programming exercise and be guaranteed they're not wasting extra time on it. And if they happen to prefer doing take home assessments, you can always let them send you an updated answer later. (But often by the time a candidate asks me if they can do that, I've already developed a favorable view of their skills and can tell them, "go for it if you want, but you've already 'passed' my test.")
By keeping the candidate-interviewer time investment the same, you guarantee that you're respecting the candidate's time as you would your own (because you're sitting there with them.) I can help them skip over the parts I'm not interested in (e.g., by feeding them info they'd be able to find via search or telling them not to worry about certain details.)
If a hiring manager doesn't respect their candidates' time, how likely are they to respect their employees' time?
In some cases we roughly timed it, scheduling an email for a time the candidate wanted and asked them to return it 3 hours later. In some cases we just treated it as an honour system. We made it clear that the task was intended to take about that much time and that spending more time was not allowed/encouraged.
In reality, we found that good candidates took ~1-2h, and in some cases where candidates spent a lot longer and owned up to it, we found no improvements. In one case a candidate submitted at 3h and then again at 8, and we marked the 8h version 1 mark lower.
When I went through the process the company was very small and Vlad personally reviewed applicants, and he used take home projects like this one. It seems like the company has grown larger so now he isn't looking at them himself anymore, but the essence seems the same. If you ever talk to him you'll find that he's basically the Hacker News commenter stereotype and his interview is really a vibe check to see if you seem cool. To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool. When the assignment says "I want a terminal-inspired email client" the project that passes with flying colors has keyboard shortcuts for everything and comes with a paragraph explaining how it never takes more than 2ms to draw a frame. I suspect the interviewer thinks you are too "enterprise-brained" to work at Kagi.
Now, is this actually a good way to screen people? A bunch of other people here have opinions on it, so I won't rehash it too much. There are obvious problems with it. Vlad basically wants people who think like him. This kind of interview has a lot of problems that have been discussed and studied at length. But if you understand that and are privileged enough to go through that process, the task will actually make sense to you. I think Kagi could do a better job at communicating this. It's unfortunate that you had to waste a bunch of your time to get to this point but I hope you can find something else that matches your working style better.
If you rely on people to work independently, you just can't hire someone who wants feedback everyday to see if they are still on the right track.
This isn't exactly the same as if I give people a test on their ability. It's different. In here you pass if you did something that scores well in a metric that you're not told about. It's not very different than if I gave you a take home assignment which is just literally "code something - you have to be good at ambiguity!" Joking, you just have to guess the thing I had in mind. It's very inconsiderate to people, and doesn't tell me anything good about this Vlad person.
> To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool.
Am I supposed to guess this much?
Do they expect me to creep on their social media to figure this out or something?
Finally, I made it abundantly clear that I am uncool, so giving me the "go ahead", simply to satisfy their curiosity... I hope they are treated the same as they treat other people.
No ill will against you, if anything you are giving me a plausible explanation where I have none. I just wish I had come across a post like mine earlier, and I hope that we, as a collective, start categorizing any job with a "take-home" as a job for the "cool kids". That way, all of us boring and experienced engineers may force companies to adapt if they actually want our labor. And those that do not adapt, we can infer the type of company that "offers" the role.
Quite frankly: yes, I am pretty sure they did. You said you "had heard about Kagi Search as a reputable company" but the point is that they make a specific product for a specific group of people and if you don't "get" that they are not going to hire you. There are a lot of startups like this, mind you, and many of them are weird cults so I understand the hesitation to spend the time learning what everyone's gimmick is. But Kagi is not one of those companies where you get in without knowing what they want. It's not Google where you can get in by knowing some algorithms or your local medical startup that just wants anyone who can write React. Notably I say this without making a value judgement of whether it is "better" or "worse" than those companies. That's just…how it is. Smaller companies, and ones that are more founder driven, tend to have more unusual and specific things they are looking for.
Re: the rest of your post; I think you deserve to be upset that you wasted your time here, and that they didn't communicate clearly with you. That much is pretty obvious. Regardless, I feel like you're still missing something here, and that makes me a little sad. Kagi is definitely not the right place for you, and they should have done better here. You are not "uncool" and take-homes are not for "cool kids". You just have a different working style or stack or whatever than what Kagi was looking for. I am sure your work is fine and that there is someone out there that thinks your project is cool and what Kagi was looking for was boring. Kagi thought my work was interesting but I assure you that a lot of people would find it incredibly tedious and uninteresting.
My goal here was to give you my view of what I think their view was. I'm not really here to condemn or praise it as there are plenty of other people doing that. I just want you to understand it, rather than it leaving you confused and upset.
When I see instructions that say “This project tests your ability not only to code, but to deal with ambiguity and open-endedness”, it actually means either a) this exercise has been so poorly conceived that we didn’t bother with a rubric or b) the work environment is so chaotic, we never clearly define specs or requirements for anything.
That, and the instruction to simultaneously “show off your skills” and deliver a pragmatic functional solution are completely at odds. Good simple solutions aren’t flashy.
What I want to see is an engineer implement an awkward bit of business logic. Does it become a million nested if statements and a "here be dragons" comment at the top, or do they identify the right patterns and build something that I can reason about when reading the code? This is far more valuable in the job, more signal in the interview, and much harder for AI to get right. It also takes less time.
I've been caught with this few times now. Spend ages trying to coerce AI to solve logic problem and end up just manually solving it myself. Whereas UIs are so good and usually near perfect from first prompt. I suspect it's the weak prompt. I need to learn and solve this before my brain completely atrophies (there must be Anthropic joke here somewhere hehe).
> Good simple solutions aren’t flashy.
Maybe showing off your skills is making a good and simple solution?
I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? I consider that a big part of what it means to be a senior software engineer.
Maybe this exact format isn’t the best way to test for it, but I don’t think “we want to see if someone can deal with ambiguity” means all the work is ambiguous. It might just mean all work needs to go through a process to take it from “ambiguous requirements” to “clearly defined specs”
Then I'd expect the interview process to focus on that process, not on the final deliverable. As it stands, the candidate tried to interact with the interviewer by trying to ask for clarification on the requirements, then making a detailed proposal, and got no actionable feedback from either.
Then they should have responded to his questions in email?
I personally love working on projects that are vaguely defined because it means getting to interact with people and understanding the heart of the problem. My favorite roles have involved reaching out to non-technical people and figuring out how to solve their problem. Often it's not the solution they even initially asked for.
But if your role is to guess about vague specs with no communication, then you're going to fail no matter how senior you are.
If you're asked to do a take home, I highly recommend confirming that there will be a follow up regardless of the assessment, and if they do not agree to this, do not do the "home work". The honest truth is that most of the teams hiring are of pretty low quality and therefore implementing a good solution is a negative because the team hiring is not at your level (which is a frustrating reason for being rejected).
I'm an early adopter of Kagi and am now planning on cancelling my account over of this. Completely unacceptable. If you don't have time to chat with a candidate about their work, don't ask them to do the work in the first place.
That’s actually a very valid point. Take-home assignments not only require a significant amount of effort from the person administering them but also from the interviewer (or rather, the hiring team). After investing the time and effort in reviewing a project, it’s reasonable to expect feedback/a response back if requested.
However, we must also acknowledge certain realities. If there are 20 solid candidates for a single open position, many capable individuals will be rejected. This doesn’t imply that they were inadequate or even “failed.” It’s simply a reality of life.
Now to be asked to justify why a hiring manager picked A and not B, legally speaking, cannot be that I had to pick one out of the amazing candidates, so I picked one. The legally correct response (unfortunately) is to get nit picky and find faults where none really exists. At least, that has been my observation in how corporate America likes to operate. And last time I checked, Kagi is based out of Palo Alto.
I would never ask 20 people to do a take-home assignment. There are so many better ways to test team fit before asking someone to commit serious, unpaid, time to a project. Historically speaking a 30 minute chat with someone provides a surprisingly good amount of information to anyone experienced in hiring.
But if you want to commit 20 people to take-home assignments, then block off 20 hours next week for 1-on-1s to chat with them about those assignments. If that sounds like too much, then don't find a better way to filter down the number of people doing homework until it feels manageable.
Why not? Does this phrasing implies any form of illegal discrimination?
And yet, the role is still advertised 1.5 months after my rejection, which means they are still getting emails from new candidates applying for the role. Do you suppose they get so many competent candidates that they can't make a decision on whom to hire?
At this point we can only speculate about their intentions, because the intentions are definitely not transparent.
Their solution seems to be nothing like "a minimal, terminal-inspired email client", and OP completely ignored the references to tools they were told to "take inspiration from"
When there is such bad misunderstanding of the requirements, I'd not waste time on the discussion either.
As someone who works at a university this is very reminiscent of students who don't understand the assignment then complain when they get an unexpected mark.
This is not that common trait. But if your company has more prototype driven approach with quick “experiment” releases some people are just very bad fit and thats bad for everyone.
There is probably a candidate that thought for 10 minutes - what best can be done in 60min. They probably leveraged some http auth and forms, run that through tiny go backend, sent it with brief email. And they most likely looked a lot better in the process.
I mean it's right there in the text itself.
What do you want to achieve here, is this a throwaway prototype or would a user ever see this? Am I going to get picked on for imperfect UX or do you just want to see something. It becomes an exercise in guessing the reviewers sensibilities.
Life isn't fair, but apparently we expect more from hiring managers. And since life isn't fair we continue to get disappointed by hiring managers.
Hiring manager:
- Gets paid to perform the interviews
- Spends a 15 minutes answering emails and reviewing the solutions
Candidate(s):
- Does not get paid for their time
- Spends anywhere between 4 to 20 hours (or more) creating the solution
This is such a lazy and convenient conclusion for someone that is giving out instructions.
Another valid reason that the students might misunderstand an assignment is because they cannot read minds and the instructions are poorly written.
Good teachers are the ones that make themselves understood by the largest amount of students. Bad teachers are the opposite.
With my best teachers, I never had to question their grading, because it was transparent from the start. So if you are getting a lot of confused students, you might be the common denominator.
Buddhist monks learn by solving riddles and guessing the messages of cryptic messages called Koans. Lets agree that students at a faculty, literally paying for tuition, should not be subject to the same practice.
That is often a problem with proposals/design docs in general. In the real job, if proposal is actually required, it would be sent it back with "please add details on UX and how you are going to store email headers". In this case, the proposal was explicitly _not_ required though, so interviewers did not want to ask for more details on the optional document. They checked what was written there, found no problems, and approved it.
Concrete and working? Technically, yes. This would have been an excellent submission for a different assignment and role. But it doesn't seem to suit the specifications for this one.
If you are asked to implement X and instead you take extra time and come back with X+Y+Z, what have you done? Wasted time e.g. money. Companies really hate wasting money.
So if in your candidate project you demonstrate a propensity for bike-shedding any task, that's gonna be a big red flag.
Edit: I've actually checked the full requirements page, and they explicitly say this in "Inspiration":
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
Also, the prompt for a terminal client really changes what they're testing! Email web UIs have been a known quantity for years. But the UX of a terminal client is still something that's not "solved." I suspect the rubric for the question has large sections about how they decide to make a terminal client that OP's submission doesn't address at all
During my college days, I had an on-campus interview with Microsoft that served as an assessment for whether I would be eligible for more in-person interviews in Seattle. The interview question was open-ended: “I would like you to design an alarm clock.” It seemed like an unusual open-ended question that prompted me to consider several critical factors, such as the intended audience, physicality, visibility, and audibility. By considering these factors and not immediately presenting a solution, the interview team informed me that I had passed the test. If they had provided me with more specific instructions or guidelines before the interview, they might have missed a signal that was important to them.
Unfortunately, I did not advance to the subsequent rounds at Microsoft. I interviewed during the fall semester. While I made it thru the full process, I was informed that I lacked confidence, but wasn’t a no. I was encouraged to reapply in the spring. The initial experience had left me drained and disheartened, so I decided not to pursue a second attempt. I now deeply regret this decision and wish I had given it another chance. If anyone is in a similar situation during their college years, I strongly advise them to avoid making my mistake. Working at a FAANG company could have changed my entire life and career trajectory.
If this applies to you, this is probably the best advice anyone can give to you. I was granted an in-person interview with Microsoft as a freshman back in the early 2000s on account of my stellar GPA. They were the company to work at back then. Unfortunately, I didn't solve the brain teaser. Disheartened, I refused to try again even when they invited me to try again the next few years, and I went with an old dinosaur company instead.
I realized my mistake shortly after starting to work at the dinosaur, and clawed my way tooth and nail into a job at Microsoft. It took me 2 years, but it was the single best thing I have done for my career, and changed the course of my life for the better.
The problem is not the lack of a "rubric", it is the sheer amount of effort expected from the candidate, especially related to the unprofessionalism of the response. This process is set up to waste a maximum amount of time from candidates without any sort of feedback, and no proof that a reward even existed.
How many dozens of people completed this assignment and got rejected? How many of those assignments were even reviewed by the manager? Did anyone actually get hired? "Giving details would make this less effective for the manager" doesn't begin to excuse this behavior.
Honestly this hiring manager casts a negative light on the entire organization. Do they treat everyone this poorly? Business partners, employees?
Somehow the only criticism I got is that it was not simple enough.
Perhaps the moral of the story is that many managers have no idea what the word "simple" implies in terms of software.
This is a classic situation:
Engineer gets asked to implement the "do what I mean" button by some manager, this is a magic button that when pressed will do whatever the manager wants at that moment. The manager acts shocked when they are told that this is not a simple request to fulfill. The manager thinks: "I am being tricked! I merely asked for a single button!"
I mean, potentially they failed the real test by asking all the questions - Kagi specifically say they're looking for people who can deal with an open-ended project, and instead of just deciding to do something cool, they seem to spend a lot of time demonstrating that they can't deal with open-endedness by asking a lot of questions and coming up with a detailed spec and asking for approval before starting.
That's not to say what Kagi is doing is good, but it's probably for the best for the candidate that they didn't get the job because it sounds like they wouldn't flourish in that kind of environment (which is better to know before starting a job, instead of starting a job you're going to hate and then burning out or failing probation)
* Don't ask clarifying questions: Get failed for making assumptions and not identifying business requirements before implementation
Anyways thanks for writing this up OP, it was an interesting read and I am a Kagi customer so I liked learning about them.
Not sure why OP makes no mention of this in their blog post - perhaps they thought those parts were unimportant? In which case I am not surprised at the results.
The project was well made, but my read on it was that they wanted to be shown something interesting. Even if it wasn't as well made of polished, I got the impression they would have preferred something "fun" or imaginative.
- Title: Make something fun for the customer.
Then your performance review:
- Manager: We cannot tell you why, but you failed at your job.
Later on I read the requirements... This person applied to a position named "Email Backend Engineer" but they actually used a third-party product (postmark + turso) for email backend! They also clearly don't think about email too much - the basic stuff, like plaintext email formatting, viewing, and folders (at least inbox + outbox) are simply missing; while there are optional features, like login screen, admin page and backend framework. And backend database does not even containing a email headers map!
That person might be a great engineer, but I don't think they would be a good fit for that specific position. I'd reject them as well.
(A separate question is that hiring manager's second response... We don't do take-home interviews, but I imagine I'd be stumped by a proposal like that, it seems so irregular in a take-home assignment. Still, I can see how the candidate took that response for an approval... perhaps a next candidate would get an extra sentence perhaps something like "actual problem grading would depend on the code quality, number of features implemented, and how close those are to requirements and job description")
Update: read original job description and not just the except from the blog. That person surely omitted some stuff in their blog! the original title was:
"The project is to build a minimal, terminal-inspired email client"
it gave examples:
"Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya."
wow! that's 100% no hire, serious inability to read the requirements.
Which requirements were not met?
- Email client can either be in the terminal (i.e. a TUI) or a web app
- Should have basic email viewing + sending functionality
Seems like both were more than met to me.
Inspiration from existing tools is subjective. I would have assumed that spending time on the UI for a backend position was not the best way to show off my skills in a take home test.
I agree that "inspiration" can be subjective, but in this particular case the solution was so much off that it'd be hard to argue otherwise. There is nothing terminal-like in it, and the tool is not usable as an email client either. Instead of doing login screens and user management, author should have made a page to view individual email or added folder (just 3, inbox/sent/trash, would already be a great improvement)
And if you want to ignore the requirements and work off position description... the position is titled "Email Backend Engineer", and the candidate's solution offloads actual email operations (storage, transfer) to third-party services. I suspect that even if candidate provided the same UX, but would have backed it with SMTP, IMAP, TCP/IP (not via http library), DNS MX lookup, resilient storage etc... then they might have passed.
But author failed to do either serious front-end work (required by question) or serious back-end work (required by job posting), so result was entirely predictable.
I get it that it's not always realistic to do, especially if the hiring manage reviews hundreds of applications for a hot role. But that's why take home assignment sucks. Some candidates may waste hours of their lives for nothing. Both sides need to be respectful of other people's time.
In the first file I opened, I got as far as here: https://github.com/Sleepful/mymail/blob/main/app/router/page...
// Create a context variable that inherits from a parent, and sets the value "test".
// Create a context key for the theme.
ctx, err := getEmails(pb, e)
Wording in the second line is not consistent with third.
I’d stop my review there. Lack of attention to detail. Author does not demonstrate an ability to think clearly.
Even after asking for all this clarity, he failed to do what was originally asked. If asking for all those details, you have to at least do the basics of what was asked.
He failed on multiple fronts here, and even wrote an entire blog post without realizing it. It seems like Kagi did the right thing by not hiring him, if we’re being brutally honest. If he would re-read the original ask with a fresh set of eyes, then look at his final product and all the communication, the feedback should go without saying. When someone misses the mark by that much, it’s takes a lot of effort to try and say it nicely to soften the blow, as a company would want to do. Multiply this by however many people applied, and there just aren’t enough hours in the day.
Now that he’s written this blog post, I wonder if he’ll have more trouble even getting past the first gate of hiring processes, as other companies won’t want to sign on to have a blog post trying to drag them through the mud over a rejection. That shows more questionable judgement.
And you are telling me that I should have spent extra time on my code comments?
I have to disagree with your point.
"Do they want to see if I can build something with as little guidance as possible?"
"Do they want me to push for more requirements like I would on a real project?"
"If I build something cool but totally off from what they expected will that make me look better or worse?"
"Or are they just trying to weed out the people that can't code at all?"
In the end I didn't get a follow-up interview and they refused to give me any feedback on how I could have done better on the take-home assessment.
Back to the OP: One such example would be to do a live code review. This could be done asynchronously or synchronously. It could allow actually talking through topics and issues that relate to the challenges in a real software project. This would allow to surface much more of the knowledge in an experienced software engineer.
I like this idea a lot.
From my understanding of the post, this was the initial screening phase as it was in response to OP's application. In other words, this is what every candidate who passes the application screen (the weakest one) is sent.
Let's say they have 100 candidates for this role. A proper code review here should take ~45 minutes to an hour. Even 15% of the candidates requesting a full code review - regardless of synchronicity - represents a 11.25-to-15 hour time commitment from the hiring team. For the initial screen. That is asinine. No proper organization would accept such a large time sink for so few candidates at this phase.
As I've said already multiple times in this thread, OP very clearly does not understand the asynchronous relationship at play here, and then based much of their interactions & interpretations on this misunderstanding.
From reading the other comments, it seems like there are a lot of mind-readers among us. ¯\_(ツ)_/¯
Yes, there were flaws in the assignment and communication with the company, but there were also flaws in your approach. You’d be better off to try to take away some lessons here rather than blaming everyone but yourself.
Frankly, if I were you I’d consider deleting both your comments here and the blog post. I don’t think they reflect well on you as a candidate for future roles.
(OP outsourced the actual "email backend", that is the thing that stores and sends/receives emails, to a third-party products. Probably not the wisest solution if you are trying to get hired to work on email backends :) )
To work in a startup you have to embrace the "just do it" mindset, you have a brain to fill the gaps, and you can always iterate later. I’d be more interested in getting the result of the take home quickly, obviously in a working state, but with a documentation explaining trade offs because of lack of time, ideas for future improvements, design choices, stuff not included (ie. a pretty UI) etc.
My take: they’re literally asking for a CRUD (if you decouple well, you can easily replace the DB with SMTP/IMAP later) with basic email features: send (to, cc, bcc), fetch, reply, reply all, transfer. That’s it, you can do it in 2 hours. Add folders, signatures, "send later", authentication, or whatever if you want to show off.
--
Additionally as an experimented Go developer (8 years working with Go as my main language), the code is not great and tbh I would have rejected it if I received this technical test where I work.
In 5 minutes of reading the code:
- lots of commented out test stuff, don't leave that in a take home or I'm gonna assume you'll do the same in your commits
- lots of println but no logging
- relative imports (eg "mymail/app/router"), don't do this
- weird choice of relatively big dependencies (turso? pocketbase? negroni?)
- no architecture or decoupling, everything is in the router
- bad use of the context (getEmails)
- no tests (no need to test everything for a take home but at least some things can be tested)
People say languages are interchangeable and you're never a "xxx" dev, but that's not true. They asked for "Proficiency in Go" and the candidate is not proficient in Go, that would be the number 1 reason for rejection.
Anyway in 2 hours I’m pretty sure I can write some Go code that would get me an offer. This is not about the amount of features, it’s about giving them what they want.
Anyway: I don't think homework assignments are a valuable mechanism for filtering out candidates. Setting aside shoulds and shouldnots and the ethics of it and everything else, it simply provides a really poor signal for the value of a potential candidate.
Especially today, when something like this particular assignment could just be pasted into any of the popular LLMs and returned as completed within a couple hours.
If an organization wants to hire developers that can take vague project descriptions and convert them into code that may or may not do whatever was in the heads of the people that wrote the project description, then (a) that's a bad practice, but (b) it would be better for everyone involved if you handled this over a video call. If your staff are too busy to do a small pile of video calls with potential candidates over a couple of weeks, then they are too busy to properly onboard a new candidate and you should probably just pack it in. (i.e., they are not too busy to do this.)
If the goal is to hire somebody that knows IMAP, SMTP, POP, etc. inside-and-out and can crank out RFC-compliant code without using any of the third-party libraries that already exist for this sort of thing, then the assignment description should have asked for that.
Homework assignments are an unfortunately common practice, but worst of all, most of them are carelessly designed.
Posting a blog article with a link to the code that they felt should have got them the job isn't asking for a code review?
That would have been significantly more in line with what the interviewer was looking for and would have produced better results. It’s a take home test. Use the tools at your disposal.
In this case, I perceive that the author (Jose) was looking for firm requirements, and did what he could to elicit them, but the Kagi folks were just looking for innovation and a demonstration of competency. They didn't care as much about the details of the submittal as they did about seeing what the candidate would do without specific requirements.
This would be acceptable in a world where time is infinite. The reality is that they are going to churn through -who knows- how many candidates, as if it was worthless, as if it was a game.
If they really wanted people to spend little time on the solution, they could include that bit of info in their instructions. Clearly, they omit that bit on purpose. Obviously this is not their first rodeo, they must have done this for a long amount of time now.
Guessing is a skill, and some people are better at it than others. An "educated guess" is based upon knowledge and experience, but little more.
I don't really know that this is what Kagi was looking for though. Perhaps they wanted something more than a guess but less than a full solution. Maybe just enough to gauge the confidence of the candidate.
The "new business" environment often involves a lot of guesswork.
- Proficiency in Go - Strong understanding of scaling and maintaining backend systems - Solid understanding of containerization technologies like Docker
Nailing the take home would require a project that demonstrates all three. Of those I think they maybe do okay on the second one, though offloading to paid services is maybe cheating a little there. I don't think they do well at either of the other two though.
From the detailed spec, one might expect the project to demonstrate those things, but the code as submitted just doesn't.
"I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
"So it is funny that my project is so weak, yet it made them update the guidelines to something stricter." - Main character syndrome. As someone who has been on the other side of these kinds of reviews, the far more likely explanation is that they kept getting submissions where the build instructions didn't work (which is not disqualifying by itself; the authors may not be on the same OS as the reviewers) and they got tired of spending time dealing with it.
Ultimately, the failure here was not a technical one but a social one. The author tried very hard to do the thing that it seemed like they were asking for, not the thing they actually wanted. The hiring manager's unwillingness to engage with the proposal doc was itself a form of communication that they were not interested in this level of detail, it was just an implicit one.
It's common for engineer types to want all that kind of communication to be explicit, and I have a lot of sympathy for those folks, but the reality is that teamwork is a skill, and the ability to suss out what a stakeholder actually wants, but isn't saying due to incomplete information and/or office politics, is a reasonable thing to select for. The ambiguity of the prompt is a feature, not a bug: it's kept the author away from a company whose communication style they're not compatible with.
(that said, this project does sound excessively complex for a take-home)
> "I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
Absolutely spot on, and you ID it later with "Main character syndrome", but it is so very clear from this post's tone & content that OP expected a symmetric outlay of effort & focus from the company's side. They thought they were the main character.
That's a fundamental misunderstanding that seems to have predicated a lot of their ultimate response: they feel as if they were entitled to much more effort from the company than they received. Such is often the case with strong entitlement, it's nearly impossible for the person suffering it to see it.
What you are saying sounds a lot like this:
"it is common for engineers to communicate properly, but people above them prefer to be vague, because plausible deniability is a political advantage to them at the detriment of the engineer"
Moreover, this exercise tells you so little about the candidate and their experience. Sure, you know they can code (or did they just use Cursor?) but you have no idea whether they are good and addressing prompts and not at making wise choices. Who cares if the program can send email? What matters is that the client doesn't choke after 10k messages are received or that unicode is handled well: that's the sort of actual challenges you'd be doing at a company like Kagi, not making enormous toy projects.
Edit: Despite that, it seems like a bold choice for the author to build a web app instead of a TUI like the instructions lay out. One could say "terminal inspired" could include web applications but I think that's a stretch.
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
Of course the OP in this post did nothing even close to that.
Yes, they should've told you not to bother after you sent that proposal. But they were completely right to ultimately reject you. You're a lot of work.
The "simpler" solution part stood out to me though. Given that Kagi was putting little effort into the thing, I wonder if his verbosity/try-hardness might have been a turnoff? Afterall, part of the thing is determining if you want to work with a person, my impression reading his blog post was.. mildly turned off? There was just a bit of desperation to it (which might be from real factors!) but still. That sounds mean and I don't like saying it, that's just kind of the vibe that hit me.
My take on the proposal: the hiring contact is not the person who wrote or will review the prompt. The candidate sent this long message and I doubt the hirer even read it. That's some nativety on the candidate's part, but the company absolutely should indicate "you're overthinking this"
The requirements to spin up and obtain live services (postmark, torso), is a bit much, especially for a take home assignment.
Personally, If your app can’t be spun up and tested with a simple "docker compose up —build", then you are doing something wrong.
My rule of thumb for take home assignments: if I spend more than a few hours on it. Then I am probably over delivering and need to rethink approach and remove shit I don’t need. There’s no way in hell I’m spending a full working day, let alone a _week_ of free labor for these assignments.
Note: also not a fan of "take home assignments" either. Of the few companies that ask for these, I consequently ask to be compensated for my time. Only 1 company agreed to compensation, but others declined and I moved on.
It can be spun with a simple "make live", I mention this in the first line of the README. ¯\_(ツ)_/¯
In the long run, venting online can be far more damaging than the initial setback - and a huge waste of time!
This was part of the assignment:
> This project tests your ability not only to code, but to deal with ambiguity and open-endedness, which is essential in R&D projects like Kagi Labs.
It was vague on purpose, they wanted to see how your decision making process goes with what you fill in the blanks with, without needing hand holding. You seemed to ignore this critical piece and went further and further against it with each message.
I cringed hard at the “proposal” he sent them (whyyy) + confusing the hiring manager as some sort of counselor + “will I get the job if I build this?”
Huge misread on OP’s part long before he wrote a line of code.
"Oh, I simply refuse to do interviews [of format xyz]. It's just not worth my time..."
I feel like such a worthless piece of crap when I read these. I apply to hundreds of places, and when someone comes along that even wants to talk to me or a miracle happens and I get to any sort of technical round, I simply DON'T HAVE THE OPTION to be turning them away for any reason.
Like, these companies have all the leverage right now, and lots of us have none. Responses like "just tell them no" seem so tone deaf. I guess some of you must be seriously baller, but it seems otherworldly to me.
This is close to how I feel when I read posts on Linkedin "demanding" that companies dispense with some unethical hiring practice (e.g. keeping up job postings for which they are not hiring, rejecting candidates without feedback, endless interview rounds, etc). I'm like, "Dude! The people who are doing this do not give a rat's ass about your preachy Linkedin post. We have no leverage here. Spare me your clickbait."
End of rant.
Look at all the third party services he uses! Nobody sane would pick that solution. That would be a completely maintainment nightmare.
He did not write an email client, he wrote a wrapper to some email client, which handles all the heavy lifting.
At the very least, write something that interact with a real (or fake) SMTP server. Other things are just cherry on top.
Seems like he did none of it, and in this company people do not talk and everyone is left guessing what is work consists in, which resonates strangely with the take home assignment directives. What an horrible workplace it would be.
The instructions seemed clear. They deliberately set an open ended task, and they want to see how well you independently add valuable features (to a well known format).
I agree that this is an obscene amount of unpaid work but if you're going to do it, you can't ask your hiring contact how to do it. It demonstrates a lack of independence, and that's something they told you they wanted for this role.
I have been at those jobs.
Trust me (or not): even if you are given this kind of freedom at first, it comes back to bite. You want to get the person in charge to agree with your design for any project that takes a week or longer. Otherwise, the moment that your solution does not create the expected happiness in the stakeholders, you are in a position to become the scapegoat.
This sounds cynical at first, but it's just reality. Besides, imagine if you were the boss of someone... It would be of immense value to get a specific plan for a large project before it even begins. There is nothing good about the uncertainty of not having a plan.
Remember this: I did not ask for the plan to be given to me, I created the plan and offered it as a proposal. I am the one doing all of the difficult pieces, and serving it on a golden platter to the person in charge.
Either someone had a vision and is saying ‘Read my mind’
The alternative explanation seems to be, there is no vision, and the interviewee needs to define it.
In either scenario, the amount of communication, feedback, specificity, lack of respect for the power dynamics is appalling.
Yes! That's exactly it.
It is mighty convenient for interviewers to feign ignorance of the power dynamics at play.
if you can't provide that then i think you haven't created one, and if you haven't created one i'm guaranteed to be wasting my time because you're just doing vibe interviewing.
Love the service. Very disappointed in the process. Four days for a take-home with absolutely minimal feedback is really bad. I decided to jump through the hoops because it seemed like a really cool company to work for.
The few that benefit from the abuse also benefit from keeping it hush hush. This is no different from obscuring the pay-bands for employees so that they can get away with underpaying the most vulnerable workers.
Just two things:
- If the candidate already has a portfolio, whether personal projects on GitHub, open source contributions, or projects they're willing to share directly, why do you need to make them implement some specific project? The whole point of a take-home assignment is the discussion that follows it where you talk about the code, which trade-offs they made and why, and you try to get a general feel for their understanding and enthusiasm for the domain. Whether they implemented something specific is irrelevant.
I get that this is often done to make lives easier for the interviewer, and to be able to relatively compare implementations between candidates, but every candidate is different, and since you want to judge their thinking the project should be relatively open-ended as well. So just spend some time to review their portfolio instead of asking them to implement the same cookie-cutter project.
- All take-home assignments should be paid. Period. Don't ask candidates to work for free. Agree on a time estimate, and pay them a fair hourly rate. The initial offer can be some average rate for the specific role, but if the candidate has a higher rate, then negotiate based on that. It's disrespectful to expect them to work for free, only to reject them afterwards. This way at least it wouldn't be a complete waste of their time.
I'm disappointed that Kagi's process doesn't take these two points into consideration.
That said, my favorite interview style for programmers is the code review. Show them a piece of intentionally buggy and incomplete code, ask them to mention any issues they find, and to fix them. This could be open ended and include performance issues, testing, etc. Then you work on it together, the interviewer almost taking a co-driver seat in a pair programming session, and the discussion is almost always relaxed and informative. I've been on both sides of this type of interview and it's always been a positive experience, and most candidates enjoy it as well. It's much better than a stressful "here's a blank whiteboard, implement this from scratch", a leet-code style puzzle, and much less of a time-investment than a take-home assignment.
Did they, though? From my reading of it, it seems like they wanted a minimal terminal inspired client akin to mutt with a fake backend and plain text.
Case-in-point: Modern job sites (cough LinkedIn). Every job listing has HUNDREDS of applicants within an hour of the job being posted. It's ridiculous. It should not be _that_ easy to apply for a job.
The outcome is what we are seeing today: A company posts a job, is inundated with 100s/1,000s of applications. In order to filter out the 80% of applicants who aren't deeply interested in the role, the company deliberately assigns busywork/road blocks to slow down the process.
The other 20% of applicants will then spend days/weeks/months in the hiring process on intro videos, take home challenges, etc. Basically anything that can be throw at the applicant that isn't time with a human.
The takeaway for each can be broken down into:
- 80% of applicants: didn't spend anything, didn't get anything, don't care
- 19% of applicants: spent time doing some/all of the busywork, _aren't_ hired, end up very frustrated at the amount of time/energy/resources that was spent only to be discarded
- The 1%: spent time doing some/all of the busywork, _are_ hired, feel great!
I'm not sure what the exact solution is, but I know that:
a) it's a race to the bottom (with the bottom being full automation on BOTH sides of the hiring process),
b) I'd much rather spend a fair bit of time putting together applications for 5 jobs and be seriously considered than spend very little time on 50 jobs only to be immediately rejected or handed an assignment before I even talk to a real human being
c) hiring teams hate the status quo, too.
If you post a JS position, you will get 1,000 or more applicants, so it is a huge amount of work behind the scenes to filter this down to try to find the vaguely worthwhile candidates to interview.
If you post, for example, a Clojure position, you will get 10s or maybe even a 100 candidates tops. And they tend to be uniformly more qualified because niche tech tends to self-select folks who want to explore outside the mainstream.
Of course, a lot of businesses want to use mainstream tech because "the hiring pool is much larger", but the flip side is "the hiring process is a lot more work", because of the volume. So, we get a crappy hiring process because they can't scale up a good hiring process :(
And, with the market flooded with experienced devs, why should any employer take a chance on a 100% green dev, when they can pick and choose from among 5+ YoE SWEs, who are desperate for jobs? As Shawn's post from yesterday showed, many of us are willing to work at much lower salaries, so even financially, it does not make sense to hire a fresher.
Anyone care to engage with this topic?
This kind of "do something for X but we won't tell you how!" looks a whole lot like market research disguised as an interview, that way they can do the research for free.
edit: commenters here are also missing the forest for the trees. If the candidate had done X, Y, Z to make a more compelling entry then it would be some other poor guy staring at a two line rejection of tens of hours of work. The process is exploitative.
I often see people applying make this same mistake with their CV. They'll have a 5 page long incredibly overfilled CV with whatever they ever worked on. Instead, just make it one single clear and concise page so that the recruiter can quickly assess your selling point relevant to the position. They don't have time for all this so why would you?
To standout, you have to be a bit creative and you must sustain yourself with your own company / startup. (4 basic instructions here) [0]
Companies do not care about you or your take home solution(s) unless you've built something that threatens their existence with a competing product that makes money and steals their users.
Stop playing by their rules with these interview 'games' unless you have lots of time. Spend your time elsewhere. You now have AI, and you should build a startup instead.
But having said this I would have likely rejected the author too. Kagi Search is a startup, these folks are typically looking for fast moving, pragmatic optimists who can strike the breadth - depth balance.
We used to have a colleague. They would collect the inputs, close themselves for a few days, come up with a solution only to learn that reqs changed in the meantime. It was not pleasant experience for anyone.
I had to submit a short written design proposal and was told to cap it at a few hours and was paid for it. It was accepted and then I spent time implementing a solution. The pay again was capped at a certain amount of hours, but I ended up going past the recommendation because I was actually having fun trying to figure it out.
I submitted the solution and after a while I got a response with basically, "it was a difficult decision, but sorry we're going to pass" and they wouldn't provide additional detail. It's been over 5 years, but I still wonder why they passed.
I can only assume they had a lot of candidates and they may have had other very strong submissions that were better than mine. However, it took a lot out of me emotionally and affected me for quite some time... more than any other interview in my ~20 year career. I feel for the OP.
A timeboxed test for a small group (say, no more than 10) of final candidates can be an excellent differentiator.
The problem here is Kagi have omitted the timebox, so less confident and/or experienced candidates go to the ends of the earth to deliver something that was never going to meet the mark.
Had Kagi said “spend no more than 4 hours on this” - and arguably reduced some of the requirements accordingly - then your man here wouldn’t have wasted a week (!) of his time and Kagi’s.
It’s a real shame that so much time was wasted by the candidate, but the requirements are very clear that “This project tests your ability … to deal with ambiguity and open-endedness”. He’d already dug a hole for himself before he’d started, unfortunately.
What if there are 300 other applicants? What's to disincentivize giving all of them the assignment even when there's only one open position? There is no guarantee that anybody will even clone your repository.
In a live coding interview, the company is committing valuable engineering resources to evaluate you. You have some assurances that they actually want to hire someone and not just get free labor or waste your time.
Your solution is the same old thing thousands of others could produce. They want more than a programmer that knows some languages, tools and libraries.
I think they might give you 10 more minutes of their consideration if you simply produced a command-line program in Go that used sockets to talk protocols to servers.
When good companies are trying to hire engineers, they don't hire based on skill alone. They also try to hire based on how a person performs given certain constraints like time and ambiguity.
Sharing a proposal and asking for more time was probably the polar opposite what they were looking for
I think a lot of product managers prefer to see simple, fast solutions to a take home assignment. Understandable and turned in on time.
I feel sorry for the author to have spent more time on actually doing the assignment after that reply.
Second - this is the wrong approach. Nothing wrong with writing a small doc. Write it, then implement the necessary features (terminal inspired client that can read and send email), and test it. Literally a cli for read and send, tack on curses or use imgui. Do IMAP/POP/SMTP or emulate it. The job position is for a backend role in the email team.
Tack on bells and whistles later.
Then I waited for two weeks, and all I got was a clearly templated response.
The worst part isn’t even the rejection — it’s the vague replies, or no reply at all.It just makes you feel like your time doesn’t matter.
Thank you for writing this. We really do need more people to speak up about these things.
But the absurdity of take home work like this is the company feels obliged to keep their requirements secret. Thus, by doing the task not knowing if it will be up to spec, you compromise on your most important skill!
1) An open-ended take-home assignment with no expected time limit and extremely little guidance is a very poor choice for an initial assessment of a candidate
2) A week of work is way too long to spend on such an assignment unless it explicitly says it expects that much time (and even then, I'd decline)
3) It doesn't seem like what OP produced in that week was terribly strong; it seems heavy on documentation and low on usable code. I see why he was passed.
4) Kagi as a company has always seemed to have an extremely fire-from-the-hip, whatever-Vlad-thinks-is-cool mentality, and I don't honestly trust it much as a result; it also seems like it therefore would not hire someone who works the way OP does.
5) Companies implicitly hire people who are similar to the ones they currently have unless they work strenuously to avoid doing so (in a personality and technical knowledge sense; this is not a comment about "DEI" or something)
6) Asking someone to put in a lot of time and effort and then providing essentially no feedback is an extremely common dick move
7) Take-home assignments should be compensated, if only some token amount, to express gratitude to the candidate for their effort. The better long-form take-home assignments do this already (Automattic is one example of a company that does this, and their other behavior aside, I think it's a good move)
All in all, nobody in this article comes off particularly well. I think OP managed to demonstrate why Kagi has this assessment while also producing an effective indictment of the assessment as a poor choice of hiring tool. Even so, I'm glad OP posted this, credit to him for showing what the hiring world is like right now. It's rough out there.
Companies are abusing their position and we should remember the most egregious of them and avoid them in the future.
Furthermore, if they are even the kind of company that thinks they're so special they can or should even ask for that, then it proves they're the kind of people I want to avoid like the plague to begin with. So when I'm asked to do a homework assignment it's actually a huge time saver for me, because it tells me right up front to avoid them.
All developers should refuse this. But there are always enough that are so desperate they'll do pretty much anything.
Remember kids "just say no"
I have been lucky enough to be able to reject take homes. Next time, I might offer to do them for a flat rate of $200/hour but working for free is something I will avoid.
If it’s going to be used as a screening process, at least give clear guidelines and evaluate it pass/fail.
The actual software engineering and coding is now a lot less valuable as compared to building something useful within constraints. Anyways, here are my thoughts
1. The interviewee failed to truly appreciate the need for ambiguity. Some people felt the interview was over after the submission of the proposal and I agree with them. Coming up with your own requirements was necessary. And this requires your judgement in determining what was essential or not. This skill is surprisingly rare.
2. Good taste is also now more important than ever. It is also hard to test but the interview questions were really good in that regard. They gave you nudges towards the ideal state and left the door open for you to determine the best "path" towards that goal. This tends to reveal a lot about a developer that they might not even be aware of - like over engineering. For a take home like this, I felt the proposal and the half a dozen AWS services was not in good taste. It was probably more complicated than it needed to be.
3. Deadline and deliverables. Exceeding the deadline is a negative. The final solution is not terminal inspired and those alone may qualify as basis of rejection.
4. Of course the hiring manager should probably have rejected the proposal but will that be fair on other candidates? I also feel giving you access to a hiring manager is a stroke of genius. They can assess you via the kind of questions you asked. And I doubt this candidate asked the "right questions". Right here means that they understood the point of the take-home and also what needed to be built. I didn't get that impression from the candidate.
5. I truly believe take-homes like this should be paid. I just don't know how it might work in the real world or whether they are sustainable. The payment here is just a consolation for those that never make it so that their valuable time will be respected. Even more so because I feel a lot of people will probably diverge from the spirit of the exercise.
Do I get some bonus points for being born in the same country as the founder?
Never heard back after submitting, spent probably 10 hours on it
This is the real take away. Don't put in outsized effort at companies that are highly subscribed. Everyone prides themselves on being "extremely competitive" and unless it's a FAANG (or well known quiet money fountain like Valve), it's probably not actually worth the effort.
If it's hot on HN, it's highly subscribed -- you just can't expect effort to be valued the same way when there are 10 applicants.
> We normally don’t provide feedback at this stage. We have had other submissions that were simpler and stronger, so we decided to continue forward with these candidates.
This would have been a great place to push and ask for a little more feedback (while being nice as possible since they have nearly no incentive to share more)... How could the other solutions have been simpler and "stronger" (whatever that means? more robust?) -- a few details on what you missed that they were looking for (tests? documentation? CI?) could at least add some value.
All that said -- finding a job is really hard right now. Wish you the best.
Make an email client, email view+send, fake backend or real imap, handle plaintext.
At this stage, for a take-home, I'd start working and write down assumptions I made as I went along. I'm probably the opposite of the author, as a take-home (unless it's the last stage or something) is, in my view, a tester to see what a person can do within a few hours of work. I've had several take-home exercises during my time as a software engineer and they varied from "we have provided all details that a stakeholder would provide" to "if you have any further clarifying questions, please get back to us".
The most recent one I took a couple of years ago came with an internal library the company used. They said, "use this library to make a web app that takes advantage of these methods inside of it; create an app that simulates behavios using those methods. Do not spend more than 10 hours on the assignment."
I started coding, I threw something together that worked in about 6-7 hours and I was writing down assumptions as I worked as well as trade-offs from those assumptions. "I assume the user would not be bothered by a failure here, if reliability would be important, what would we want to do in the case of a failure? Retry? Back off retries? etc etc". I then provided the code and provided a list of improvements "Add unit tests to these 3 components, add integration test to ensure this functionality works end-to-end if its essential, improve UI, clean up code base, refactor these services, use a framework/library for the UI instead of hard-coded JS to make these few calls. I wrapped it up because I think I was 70-80% there."
During the next interview with the architect of the company, he said that the solution I provided worked fine, we discussed some things I did and he asked why I did them, but specifically, and this is why I'm commenting here, he mentioned that he appreciated the assumptions document, the future improvements and the "I stopped here because I'd rather get feedback on this and refine, as opposed to keep building something I imagine you want". He said that the ability to work up to a point where you hit the majority of the story and then get feedback on that incomplete project is better than having someone dissapear into a cave for a month to try and come back with the absolute finished product in their oppinion and then having to make changes to get it in line with what was actually wanted.
So I guess, become comfortable with uncertainty. If nothing else, ask only "hey, I assume you want something along these lines that I can bang out in 5-6-10-20-40 hours. If you're not happy with what you get for 5 hours, it might not be a good fit in general or if you think I prioritized the wrong thing in those 5 hours, then we can chat about that too". I am also saying this, because my current role is a lot more in line with what the author is looking for - they spend weeks refining requirements, they write documentation, they create mocks and they have meetings over meetings before I even know what I'm supposed to do and within a day or two they start realizing that what they described is about 10-20% off what they wanted or what is possible and the whole cycle of meetings starts again. Instead, and I've been pushing for this each sprint, I'm asking them to accept a certain level of unknown in a given story, we work on it, we see it behave and get used and refine it based off of that.
The author seems to want waterfall, but agile exists for a reason. Hell, let's not even call it agile or whatever else, in a real situation, you start doing and you learn as you move along. You refine based on feedback, you refine based on new experiences and based on new requirements. You work in the murky areas of someone elses mind. Or not, I don't know, but expecting a series of jira tickets with screenshots and deliverables from a company that just wants to see how you think, how you work with uncertainty and how you deal with unknowns feels... wrong.
They literally said they want a simple TUI email app
> If he wanted a simpler solution, he could have told me on March 18, when I submit my proposal.
When he said your emails could be stored in a fake database or loaded in memory it was obvious they didn’t want something over complicated.
No one likes over engineering, it can come off as egotistical, indulgent and more importantly a sign of technical debt lurking ahead.
Your proposal is long winded, you’re not the only applicant these people are looking at my guy.
I’m not sure what Fargate is (nor care) or even why you suggested AWS and all that other jazz but you clearly went overboard. No one wants a developer where you ask them for A and they return A B and a half finished C.
Yes, you’re allowed to ask questions but the ones you asked showed signs of overthinking and lack of initiative.