I've used both. It depends on how personal your personal email is. A custom domain is only worth the expense and trouble if your email address is in any way part of your first impression.
If you are just giving this address to the services you sign up for, friends and family then GMail/Outlook's reliability actually make them preferable to setting up an email server on your domain.
Fastmail et al sell their paid service on a promise that you see no ads, and the contents of your emails will not be scanned by the same automated services that Gmail/Outlook use. This doesn't translate into better sending/receiving of email and depending on who you are, you can decide how much that extra privacy is worth in terms of dollars.