On the other hand, if you absolutely depend on the ability to send emails such that your recipients reliably get them, hosting your own email server is extremely tricky. One could carefully go through the checklist of SPF, DKIM, ip blacklists, etc and emails will still be rejected by MS Hotmail, GMail, Yahoo, etc. Those giants are "black boxes" when it comes to their heuristics for rejecting incoming mail as spam. E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". Trying to debug your "sender reputation" is not easy.
In the 1990s, I hosted my own home email server over ISDN lines. These days, I have a million other things I'd rather do than babysit a personal email server with software updates, SpamAssassin lists, etc. I get the whole decentralized ethos but it's just not worth the effort for email servers.
Running your own mail server does require investing some time in learning the software, the protocols and all the ways spam filters will sabotage your attempts to send legitimate, non-bulk e-mail, as well as a little time for maintenance, but it's really not as bad as people say, at least in my experience. It's not something you "just" do, but it is doable.
I don't think we should dissuade people from doing it, especially if the fact that more people doing it means that it'll be easier next time because it'll be slightly more common. Many of us are in tech and the field is a small subset of the population. Even if it's a small amount of servers setup by us, that could make a noticeable on those working at bigcorps who write the hostile receivers.
Setting up is easy, maintaining and responding to all the blocks you may get is ongoing and simply cumbersome. That is what people are referring to when they say its tricky.
You may wake up one morning, and find your host is blocked through association (if you are lucky - some hosts will simply silently swallow your mail - i.e. it never arrives and you dont know about it). You can apply for removal in some cases (and wait for it to update before you resend), and in other cases find your hosts entire subnet is blocked, resulting in you having to set up another server on a different subnet in order to relay your email.
There are ways around a lot of this, but hosting on AWS/DO/Etc not having your own assigned subnet, etc, will most definitely result in the above when you least want it to happen.
Of course SPF and DKIM were supposed to alleviate the need for these IP based blocks, but the reality is that they haven't changed a thing when it comes to sending email to large hosts.
Thats why we say its tricky.
You're not just setting up mail, you're dealing with an email cartel that does not have any desire for small players to exist. Note that I said "small players", not "spammers". Enough spam makes it to its destination that there are certainly palms being greased.
You could attempt to limit the outgoing emails per account per hour, however if that's set too low then you end up with other users who can't send out emails to their "mailing lists" consisting of hundreds of contacts, instead of using a real mailing list to manage it.
The effort just isn't worth it.
Is this an urban legend that just keeps getting repeated?
I don't know if anything like SPEWS is still around but they used to do large blocks. I still don't know if it was genuine attempt to reduce spam, or performance art, or god-tier trolling.
https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_...
https://web-beta.archive.org/web/20030609200813/http://www.s...
That’s why
A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.
Also, it isn't in said idiot's interest to deliver your email properly. They would rather fart around with machine learning and blame whatever happens on incomplete training data...
You can do "everything right" rolling your own email, but you are still relying on the receiving parties to do their job and implement sane spam prevention policies. This is far from what actually happens.
I lead development on a project that sends a lot of (legitimate, user-requested, transactional) email to many mailboxes belonging to a certain government department. I used Amazon SES as they have a pretty good track record. The receiving mail servers were configured to use a few overzealous blacklists that were causing my email to be silently swallowed (and reported as delivered). This was based on an IP black-level ban. Simply Googling the names of these blacklists showed report after report after report of them being completely useless, with many seasoned sysadmins saying that nobody worth their salt should pay any attention to them.
This was extremely hard to troubleshoot as this department was not a direct client and was of limited helpfulness. It became apparent that using SES's newly-launched dedicated IP functionality wouldn't have helped. I made full use of my AWS Business Support, the multiple people I talked to made it clear that SES put a great deal of effort into ensuring their IP ranges are "clean" - but there are some spam lists that do not play ball. The most important takeaway was that I was doing everything right. I was using a reputable mail provider who have very stringent spam prevention controls. I utilised SPF and DKIM. The content of my mail shouldn't have (and didn't) trigger a content-based spam detector. It was because the people on the other side weren't doing their job to an acceptable standard. When I was eventually able to open a channel of contact with the receiving department, they confirmed it was an IP range-based ban.
The fact that I was doing everything in my power to do the right thing had no effect on the situation. I still wasn't getting mail to customers. Saying "darm, these sysadmins really aren't doing their job properly!" had no effect on the situation at all.
We still live in the age of almost-but-not-quite-plug-and-play email solutions like Exchange that can be configured by people that probably shouldn't be configuring mail servers. For all I know, there was a list of blacklists on some Hosted Outlook control panel and the person in charge ticked all of them because more is better. As long as this is the reality I live in, I'm going to do all I can to play ball and succumb to the black magic. I can imagine my experience would've been a lot worse if I had been operating a mail server out of a dirtier IP range (generic cloud hosting provider, retail ISP customer IP range.etc). If I recall correctly, AWS severely limits sending mail from EC2 instances because they don't want their IP addresses / IP ranges to end up on mail blacklists.
Step one to freedom was to use my own domain, redirected to my old account, but I'm seriously thinking about doing it all myself.
But keeping a whole server up to date, secure, etc.... that's a full time job. Is there a good solution?
I believe they stopped this recently.
But you could have always become a paying customer of Google.
I recently signed up for ProtonMail.
I wanted a reasonably cheap and secure email service that allowed me to use custom domains. Proton Mail seemed like a good fit. No complaints so far.
One other option that I almost went with was Rackspace Webmail:
https://www.rackspace.com/en-gb/email-hosting/webmail
However, that $2 per user per month only applies if you buy a minimum of 5 user licences, so it worked out more expensive for me than the 5 Euros a month for ProtonMail Plus:
The moment Yahoo, Microsoft or google decide that they don’t like you, you’re SOL.
Yahoo are the worst. If you try and deliver to them you get a deferral with an error message in your log with a URL. Then you have to open the link in the URL and fill in a form. They don’t have to accept the form and they ignore you for 3 months if it goes wrong.
This happens even if you’re not on an RBL and have set up DKIM and SPF properly.
Edit: you want to see the trouble we had to go to so we could run an SMTP server in AWS for outbound/abuse address inbound only and get that talking to Office 365 for internal use only. Two days of hell.
I've only seen one blocked send happen -- blocked by my grandma's @att.net account. Since it happens so infrequently and nobody uses @att.net, I just re-sent from a Hotmail account instead. No issues with the other major players. But for my use-case it's easy to mitigate and if the problem persists I can invest more time in it, but one recipient blocking me in 4 years isn't bad.
It's the only way to have ownership, which is is one of the benefits I really like - Google, Yahoo, etc. still get pieces of my personal email history because nobody else self-hosts or uses PGP, which is disappointing, but I prefer it over handing one player ownership the full history.
BTW, I'm running it on the same 512MB DigitalOcean droplet that I use to host my static sites (personal website, small product sites, etc), so it's basically free since I'd need to host those things anyway, which is nice. Needs some swap though.
Edit: Not saying these points are invalid. They're certainly valid, a service like Gmail _will_ be more reliable and easier. If you're blocked for some reason or have any other email probs, there's nobody else to fix it besides you.
BTW, AT&T's customer email is now hosted by Yahoo.
MS has 2 layers of spam blocking - one for Outlook.com, and a much stricter invisible layer for O365 with no support team.
It’s setup to receive everything that’s sent to it, which means I occasionally have to delete rando spam. TLS is setup too. But it’s an interesting system because you can keep tabs on what exactly you are receiving per service (e.g. using instagram@mydomain.com), and maybe one day will tip me off to services giving away email addresses.
Otherwise I’m using gmail for personal sending.
By the way, here's the obligatory HN-style critic of the format, not the content: the blog template in this is a little annoying, it fiddles with scrolling by making it really slow on Safari on Mac, and Reader mode doesn't work.
Yeah thanks for the feedback, I need to get around to self-hosting my blog as well. I set up blogger a few years ago and I never got around to changing it.
I also ran through a couple of IPs with my provider before I found one that wasn't on any meaningful RBLs (my IP's on SpamGrouper, but that list is clearly run by an insane person and nobody seems to use it).
As other people point out, some mail providers are just complete assholes and will blackhole your mail with no indication to the sender or recipient that it happened.
A reasonably good and cheap service I could not recommend enough is https://www.migadu.com . They allow you to use unlimited email domains, storage, addresses with the only limit being on total daily outgoing emails. The mini plan allows 100 outgoing emails a day which is more than sufficient for most of my purposes.
Hosted on a cheap Strato VServer in Germany, I've never cared for the technical details, could not explain right now what DKIM and SPF are (and they're not configured), and only recently installed a self-signed SSL certificate in my Exim configuration to be able to use it with a Desktop client for submission (pretty sure outbound traffic still runs unencrypted).
To me it falls under the same category as assembling my own computer. I can do it but to me it's not worth the trouble.
Between DKIM, DMARC, and SPF, security, backup strategy, the fact if you are an open relay for even a day a bot net will find you and get your IP blacklisted for life... or an ISP could just blacklist you because they saw other spam from your same subnet on a shared hosting provider...
Granted this article covers a lot of that (it talks about DKIM, DMARC, and SPF) I'm still counting this as one of the things I outsource.
Even at the hundreds level, I'm sure there are people who'd rather outsource. It's not just email that these guys (Gmail, Outlook, Fastmail, etc) provide.
We used this to setup a multi-domain email server. Works reliably and fast at both sending and receiving mail.
