A "side project" is a project, which by definition, doesn't have my full attention. I need to have confidence that I can tweak it and re-deploy it when I have a moment of inspiration or a bug-report.
The main way to be confident that you haven't broken things? Adding test-cases. I started out writing unit-tests for the APIs I present and consume, then later added more tests which run from "Signup, Use the service, Delete the account".
In all honesty I don't run the full suite very often, but when I do I'm glad I took the time to write it.
It's a great sign people are writing automated tests for their side projects because based on my experience there are still many teams that aren't bothering to do so for bigger production projects adequately. It's one of those things everyone knows they should do and talk about it but the resources and priorities are almost never in place. The test results need to be visible by more than just the testers too.
I have a particular interest in test automation due to my involvement at Tesults (https://www.tesults.com). This is a test results dashboard for teams of about 10-20 or more - it's not designed for tiny teams or side projects but if you're interested in trying it out for a small team send an email using the address on the site with the link to this page on HN and we can sort you out - only really worth it if you are doing continuous build/integration too, overkill for one or two person teams.
Call it post-facto regret I guess.