IP range is definitely a problem. I manage to get blacklisted a couple times per year by Hotmail, who still only support IPv4, despite SNDS always saying everything is fine. They don’t seem to care that blocking a whole IPv4 netrange of a reputable VPS provider because some other customer was sending junk mail will cause collateral damage.

IP/domain reputation is also probably an issue; I’ve been using the same IP address and domain names for over a decade so services like Talos are reasonably satisfied I’m harmless. If someone sets up a server and it has never sent mail for their domain before—or, worse, they get a radioactive IP address—it may be hard to become accepted by other email providers.

I get the sense that the relative lack of trouble I have maintaining a mail server is more to do with being effectively grandfathered in and worry that if I ever have to shift away from my current provider that email sending is going to become a lot harder since I won’t have my IP reputation any more.

It's possible to self host.

It's impossible to self host with the guarantee most emails sent from that host won't be outright rejected or flagged as spams by centralised mail services.