We ideally want someone who is
1. Adept at good architectural and design patterns.
2. Understands library internals and performance nuances.
3. Can scale a complex codebase with software engineering practices (Test cases, CI-CD, migrations, etc)
4. Can mentor the team to better themselves.
How do you test for all these?
Note : None of us currently on the Frontend team have over 4 years of exp. The person coming in will be leading the team.
First, accept that the vast majority of people you want to hire are already employed and normally couldn't be bothered with your startup. The leftovers are either really unlucky or not worth hiring.
Second, go after the top 98% and leave the others for the rest of the industry. Don't waste time trying to interview them beyond a quick phone screen on personality, because any time of theirs you ask for is extremely expensive to them and you need to acknowledge it from the start (or they'll not respond). Offer them _more_ than they're earning now. After you find one you like, and you're willing to pay them more than they're currently getting, and they're willing to jump ship for that, close the deal as quickly as possible.
This works because their current employer vouches for them (by employing them), and their current employer is presumably a better judge of technical skill than your team is. So leverage that skill instead of cobbling together some facsimile yourselves.
The downside for this approach is that if they're currently earning $200k and you have a budget of $120k for the position (or even $220k), you're not going to hire anybody. It's a shared delusion in the industry that everybody can lowball the senior talent and they won't wise up.
It's been said before, but it bears repeating. For senior talent, you're not buying the job, you're selling it. It's a buyer's market and you need to play motivated salesman and not choosy PITA customer.
Hopefully you mean the top 2%, or the 98th percentile. /snark
Generally though, I agree with this advice for small companies (i.e. your first ~5 hires). I would recommend making a list of experts on your stack who you would love to work with. Trawl GitHub, StackOverflow, HN, LinkedIn until you identify a list of candidates that are verifiably good at what they do. Then just start recruiting them, a.k.a. selling your company and vision.
For small companies, this aggressive approach seems like it would be both more reliable and efficient than the "pull based" approach of vacuuming up resumes.
Hey, I'm not employed! Haven't had a full time job in almost a year. But it's mostly because working 10 hours a week on projects pays as well as a full time job.
If someone wants to hire me, I'll listen, but you probably have to offer double what I make working from home.
Why? There is a lot of stuff happening in the top layer of everything (UI, full-stack, os/cloud/vm/container)... and it is impossible to evaluate properly knowledge in that area.
BUT... If I ask for basics (OS, Database, organization, how to isolate processes from each other, how to scale), I can see if the answer is solid technically, if the candidate can articulate an answer, how she/he communicates.
Also, I always ask for a projects candidate is proud of.
I want to hear what (s)he did, and why. Architects have to decide stuff, so I want to hear the whys and the why nots.
Then I ask what the would do differently now in the same project.
Reject the candidate if (s)he would do everything the same way: there is always room for improvement, and shit usually happens.
This is a sign other people did the work, or the candidate has a complicated personality that might backfire after hiring.
(edit: sorry if the answer looks a bit harsh)
What was the worst problem erformance wise he had to overcome? What was the largest team he ever supervised, why did he get in that position? If you talk about these kind of questions for a good hour or two you already will feel something about this person. You then have to transorm this feeling to a yes or no as a team. As he will be your future mentor and technical leader I would interpret the "yes,but..." And "no, but..." Answers as "no".
Grade the candidate on each trait based on how well they answered the related questions and pick the best one.
Because you know the questions you can prepare and not be intimidated to ask someone more senior.
For instance:
How to design an app that feeds from two radically different apis? (no need for detailed code)
What perfomance practices they used in library/framework/their own code and why?
What test cases they would write for (a simple) given spec?
Ask them about instances were they mentored/helped/coached teammates.
Good luck.
(I am not affiliated with them in any way)