Since you asked for anecdote from someone in any field, I'll give you my thoughts as another SWE.
I quit my job to work on something I care about, and it has been the best decision of my life so far. There are no directions or signs to guide me, besides just trying and finding out.
Juggling a job, life stuff, and trying to fit in time for potentially life changing stuff like this is not a good idea imo.
I always crushed it at work (EE in every performance review, paid highest band of salary, etc.), and have chased excellence in everything I tried since some of my biggest failures in my early life which set me back years. Anyway...
Having a job is the average case. If you believe in yourself, and know that you are competent, do not not worry. Do you have savings (big tech? I'm sure you do) and a plan?
Go for it if you have savings and a plan. You can always find a job, there will always be jobs out there for people like us. If it works out, you will be happy. Don't take advice from risk averse people if you're capable!
Either way, if you feel deeply about something you should go for it and figure it out. Good luck!