You seem to be under the impression a side-project means spending every spare time working on it.
> There is a risk though that after ten years you could be a great programmer with a great portfolio, but with a failing body and unhappy social life.
There's no reason this should be the case, and life is full of risks.