I agree. Self hosted my email for over a decade and as long as you do the recommended SPF, DKIM and DMARC you have basically no problems.
Occasionally I turn up in spam when I email someone I haven't before, but that's usually due to the .ro in my domain and the forced text mode rather than html.
I've found problems only start occuring when you send transactional email. User signups, notifications, etc. Anything really automated.
I think people like to joke on self hosting email because there are a lot of moving parts and it was hard to diagnose why something went wrong. Until the last decade and early 2010s there were no all in one self hosted solutions that made it easy & available for most to do it.