I do it as a challenge for myself and a way to give back to the people I want to help. To me there is nothing more exciting than having to potential to shift the paradigm.
I have a well paying job now, but would drop it once my startup launches even though I have no plans or real ideas for pulling in revenue (income). But I have faith that the more people I help, eventually money will come.
I agree with your thinking. Do what you feel is right... there will always be startups of 10+ people for you to join later on if you so desire. If you join now, you will always be wondering "what if..."