I think this is a bit of a depressing take. Even committed parents and spouses take alone time for themselves and have hobbies. Just treat your side project as your hobby and you can still have a healthy family.
Also, think about how much time the average person wastes day-to-day. Total up all the time watching television, reading articles, browsing online. Outsource your chores and improve the efficiency of your day job (e.g., work remotely to eliminate a commute) and you'll have plenty of time left over for a side project.
You might say — how can a side project be successful on 5 hours a day in the early morning? — but consider that someone pursuing a startup for 10 hours a day isn't going to have a significantly higher probability of success relative to other factors that make a larger difference (ability to interact well with other people, product-market fit, etc.) Sure, you might reduce your probability of success a bit, but you are essentially buying a lottery ticket anyway when you decide to start your own company, so would it really bother you much if your already small odds of success are 60% of what they would have been if you committed to the startup full-time?