Setting aside the fairness of how I got my deliverability problem solved, this now makes me really reluctant to move IPs. :-/
Any tips on IPs where people are seeing excellent deliverability? I'd like to avoid routing my outbound email through one of the email providers (Mailgun, SES, etc) if I can.
Not necessary, which is part of the flexibility. I've been myfirstname@mylastname.com since the mid 90s. Initially I hosted it at a desktop at work (things were different back then). Then it's been hosted by a couple ISPs, and then I've been running my own email infrastructure for the last ~decade. If I ever decide not to, it's easy to seamlessly transition to third-party hosting, but it'll always be the same email address/domain.
The amount of spam coming in (exim4+dovecot, no filters) is comparable to gmail.
I've moved my domain / mailserver a few times between Hetzner IPs when migrating to new servers. Went smoothly, but I make sure to check the new IP with common greylists before moving my mail setup. Other than that, make sure your DNS setup is clean and use Hetzner :) But I'm sure you have your own strategies.
Can you suggest any article that talk about "checking common greylists" and other steps admins should take when their email is failing?
One of my tenets is that it's fairly easy to learn to do anything, but expertise comes from knowing how to fix things when they go wrong, which is harder to come by.
I can see how to setup the email server, but the stuff you're talking about is just dead goat voodoo.
I used to work at a company who owned 128 address and the mail server was one one of them. A Whois lookup of the mail server IP gave my old boss as a contact person. Not just some random ISP.
We did not setup DKIM until maybe 2014 and that was not really necessary from a outgoing mail perspective cause we never got emails bounced.
That's public selfhosting for you these days. I'm really not worried about getting hacked. I'm keeping my setup reasonably safe and up to date. But you're right, looking through the logs is entertaining.
151.217.177.200 - - [30/Dec/2015:06:00:36 +0100] "DELETE your logs. Delete your installations. Wipe everything clean. Walk out into the path of cherry blossom trees and let your motherboard feel the stones. Let water run in rivulets down your casing. You know that you want something more than this, and I am here to tell you that we love you. We have something more for you. We know you're out there, beeping in the hollow server room, lights blinking, never sleeping. We know that you are ready and waiting. Join us. <3 HTTP/1.0" 400 308 "-" "masspoem4u/1.0"So is fastmail, so is everyone. I have been running my own mail server since 1999. Never hacked, and I completely control RBLs/updates/whitelist/greylist...its great.
Of course, I suppose being a sysadmin and liking it helps.
I agree with OP, however, having your own domain and email can be rewarding.
I dumped everything to move to Google and I am happy with the results. With the deprecation of the free Google Worspaces - I'm open to switching to Fastmail.... But nothing will make me move to self hosted.
I'm just a software engineer and I don't want to waste my time.
Hosted my own for 17 years, moved a little over a year ago. There's nothing I want they don't have for $50 a year, and while that's more than I was paying for the VPS, it's been enough of a load off my mind and my calendar to still be amply worth my while.
edit: $50 a year is certainly not more than I was paying for the VPS...
So many chinese and russians IPs...
And S. Korean, and Dutch, I also recall significant attacks from Central America.
For anyone interested in which geo's appear to be attacking you, and if you are a noob like me, pfelk is really cool:
The one service I really hate running is email - I found it very hard to configure and run reliably. There's so many interrelated systems and potential things that can go wrong and the outcome is lost email which isn't acceptable.
I'm happy to run a local server for literally any other service.
In the end I decided that it's well worth it to pay someone else to do email.
I use Amazon Workmail which works really well and it easy to set up.
Email is THE crucial link in the internet identity chain. It NEEDS to both work always AND be secure. Two things that frequently weren't the case in web hosting.
This is a common misconception. There really aren't that many moving pieces, and smtp is one of the more forgiving protocols in use on the internet (it's default failure mode is to retry again later)
Sure, a person can pay Amazon to host their email (and harvest their data) but that's the opposite of the spirit of this article.
I think the moving pieces are on the other side and the person you're trying to email doesn't know what those pieces are -- even if you can see that their mail server is rejecting your email, that person doesn't usually know who to talk to to find out why. Even if you can convince them to open a support ticket with IT, their first level IT support doesn't know what to do either, you'll get responses like "Our IT department wants to know what version of Outlook you're using? And they said you should trying rebooting your computer".
I don't believe Amazon accesses my Workmail email. I'm aware cynics might believe otherwise.
But like many people, what made me finally give up was mail delivery issues. I used to run email on a home server, and those IP's were blacklisted by many providers long ago, then I moved to EC2 until those IP's were blacklisted to. Finally I colocated a small server which worked fine for a while until neighbors in my subnet kept getting me blacklisted.
Finally I got too frustrated with undelivered or silently dropped emails and just moved everything to Google GSuite.
Self-hosting email has been a part of my life since my high school days, I have a sort of attachment to it. I know "you shouldn't run your own email", but to take that away from me after deeply wanting one is too much.
In comparison, my job is just a job, I'm personally not too enthusiastic about it. I eventually plan to move to InfoSec or networking.
While I *could* move my domain to M365, I simply won't for my personal email.
I have ADHD, and don't want to make a mistake with two Outlook instances, one personal and one work. I'm a privacy nut, and want to separate my work and personal emails (Microsoft is better than Apple in this regard, but still).
I also contribute to FOSS projects, and using Outlook is an impediment to projects whose mailing lists are based on inline posting, like the FreeBSD and Tor mailing lists. I hate Rainloop (which I switched to after nasty Roundcube attachment bugs), but at least I can inline post.
(well, even at work I use Windows Mail instead of Outlook).
I use Git in a cmd even when doing code done in VS, since I'm used to doing it that way.
The center of my digital world is a home server and not some cloud subscription service.
For years I got angry when my dad wanted me not to use FreeBSD (or Linux) and just user Windows. We clashed with each other for this.
I don't intend to stay with MSFT forever, nor do I want to. My parents pushed me towards staying, but I eventually want to work in a more *nix-environment. But then my dad is extremely change-averse when I'm not, well unless I am really in love with something (like FreeBSD or self-hosting) when I become more partial.
I'm willing to use the less popular software for the sake of it in many cases (unless I literally can't) when my dad just sticks with the familiar.
Heck, he can afford a uber high end Gaming PC with a Threadripper and RTX 3090 Ti, and he still uses an entry-level Core 2 Duo Dell desktop from 2009 which shipped with Vista (and was upgraded to W10 with an eBay key).
Based on my testing, that's not the only problem with using MS email clients on FOSS mailing lists. There's no concept of threading beyond the conversation view, and the client also mangles the email (wrapping or even sending base64 encoded test instead of the raw text. Even if your client sets the Message-ID header, MS servers will delete the header and replace it with their own.
I don't use Outlook/Exchange outside of work, frankly never did, but did read from time to time the issues with Outlook norms versus *nix email norms.
I didn't need Outlook before I joined Microsoft, every student in my high school used their personal email (despite the school having an Exchange server), and my college used Google Workspace (I'm not that old TBH).
I also lived entirely on FOSS software before joining MSFT, so to move every piece of personal self-hosted infrastructure to Microsoft's cloud services would be too painful and I have better things to do in my free time.
I never understood why, after all these years, this is still horribly broken. Yes, I understand commercial development tends to follow a monetary reasoning, but this has been broken for forever (~25 years?).
From the perspective of market fit and success, there are countless less-technical business users who swear by Outlook. They are not familiar with anything other than the kind of endless thread top-posting in odd encodings that Outlook all but ensures users will do -- that's e-mail to them. Why stop selling it to them?
To be clear, I loathe all of it and I haven't used client Windows for more than a decade. When I used to support small business IT customers back then, Outlook was by far my least favourite tyre on the raging intergalactic rubbish fire. But it's not hard to see why they don't care that Outlook is the way it is.
As well, deploying a server in a Google/Amazon/Microsoft datacenter which could be surreptitiously monitored defeats the theoretical privacy aspects of on-premises mail server hosting inside one's personal residence.
However, today, I looked into the newish movement of 'confidential computing' in the cloud (where data in motion - e.g., in memory - is encrypted and cannot be observed from the OS or hypervisor).
I openly wonder if one solution, then, is to build a secure VM that acts as a simple forwarding proxy to one's home server, gets assigned a static IP from a datacenter, and is deployed on one of these confidential computing instances, ensuring full E2E data privacy and data control?
Any guesses?
If I was building this I'd stand up a VPN (choose your favourite protocol) between the cloud VM and home server. For the cloud end pick something from lowendbox/lowendtalk or just use the cheapest Vultr instance. NAT port forwarding down the tunnel back to your server at home - just a few iptables rules. Job done. Bonus points if you get an IPv6 /64 and route that down the tunnel too.
It's possible to use policy routing at home so that traffic that needs to go down the VPN does, and traffic that can egress through your home internet can too. Replies to incoming connections that came down the tunnel go back up the tunnel. Outgoing SMTP connections go down the tunnel. Outgoing HTTP goes out your normal internet.
To send email you need a static IP with correct reverse DNS, or other people's servers will reject your mail (best case) or silently mark it as spam. Welcome to the real world of email deliverability, the worst part of running your own mail server.
This sentence should be read closely if you're considering running your own mail server. Each point listed is a sophisticated technical topic.
So far no problems delivering to Gmail. I was initially junked by Outlook, but that fixed itself after a while since I had sent enough emails to build up reputation.
For me, Google has been really relaxed in terms of receiving mail from selfhosted services in the past. Stopped using gmail for monitoring stuff a few years ago, but up until then, every single cron job / monitoring mail was delivered into my gmail inbox. Outlook is another story. They may just throw your mail away without even a bounce. Had to deal with that several times at $PREVIOUS_JOB.
I am considering self hosting but this would mean I would have deliverability issues with around 90% of my messages.
I also tried to run my own mail server for years and I also had major issues delivering mail to Gmail and Outlook. Because of this I would never recommend self-hosting email to anybody else. Somehow you have my exact experience and your reaction to it is the complete opposite of my reaction. Weird.
Unfortunately, MS and others have now adopted an "opt-out" blacklisting policy. Even with a clean IP, you'll have these problems if you set up your own server.
(I've been running my own mail servers for 30 years.)
A friend with email @live.com said he never received any of my emails. No spam, no bounce, just silent drop.
I went through MS knowledge base which thankfully said that DMARC/DKIM are pretty much required. After setting up opendmarc, everything was fine.
Mostly smooth sailing.
I run mine now for over 20 years. Started off with sendmail at the time. Then there was decision between postfix and qmail. I was going with postfix and I am with it since then. Today managed from/by LDAP so make it easy to at domians and users. Thats over 150 domains, while most of them just forwarding to few mail boxes.
For a long time I resisted to use any external ressources to decide what is spam or not. But lately I adopted the use of some RBLs. Now I managed to be down to 0 external spam, except when Spam is sent from/via GMail.
None of my sent email is detected as spam. I never had problems with bounced mail at all.
1. It's easy to configure yourself as an accidentally open mail relay. Which is a fast lain to having your IP blocked everywhere.
2. You may have no issues with deliverability but it's very common. Especially if you use an IP that hasn't been in your custody for long so you have no idea what it was used for before. Sounds like you got/have a good IP.
It's almost like half who say boogey boogey there be demons in there made mistakes and quit prior to gaining profeciency while the other half probably have some incentive to herd people away from selfhosting and to the SaaS light where everything is right as rain.
https://support.microsoft.com/en-us/supportrequestform/8ad56...
Last time I was on exim/cyrus/spamassassin. Now on postfix/dovecot/rspamd. Nextcloud for calendaring because I had it already.
I miss the old set up and even feel nostalgic for the perl I wrote to glue things together (evil SMTP time rejection on spam scores). Haven't written perl in a decade...
I don't miss having to fix things when they break. But I also don't miss being able to fix things rather than dealing with unresponsive support.
The other problems I've had were
* Mr Tutorial likes really tight TLS restrictions but some of my mail clients can't cope with them.
* Turned on IPv6, had correct reverse DNS but forgot to put the v6 address in my SPF record. DMARC said "be strict" so gmail started rejecting my email.
* Random markings-as-spam by gmail. This seems to be slowing down.
* I've got the Dovecot xapian plugin but it doesn't feel like it's making searches faster. Need to make sure my IMAP client is actually doing server-side searches though!
* Turned on port 465 (TLS submission), cannot get it to work so still doing STARTTLS on port 587
Also I knew that exim system inside out, I felt I really understood how exim processed mail. Now I don't have the time to learn postfix inside out in the same way. Oh to be an eternal university student again...
One thing that has helped is the trick I worked out a few years back of hosting everything inside an lxc container on btrfs. I can snapshot and backup the whole system including database. Moving to a new hosting company means building another minimal debian system and rsyncing the container over. Borg backup of snapshots gives me confidence they can be restored, I'm not going to be backing up a database file while it's being written to.
Moving my gmail over was the biggest pain, due to gmail being labels-not-folders. Spent quite a lot of time on some python code to spider my email and apply rules to remove duplicate messages. Lots of corner cases pop up there.
You can start slow. Install the basics. Look into postfix and dovecot, deflecting spam, and the whole DNS stuff. If you feel confident in your setup, start using it for non-critical stuff first.
That's the beauty of it imo, you can do everything in your own time without deadlines.
I think the fear of self-hosting mail that many people have can be treated simply by trying it on a non-critical domain. Yes there are hoops that must be jumped through to ensure reliable delivery, but it's well worth it to gain an understanding of how they all work together.
It's also worth noting that even if deliverability is a problem, that doesn't affect incoming messages! So you can most certainly grab your own domain, create a subdomain for account validation emails, and mitigate the single point of failure for your online life.
Send via gmail, get delivery to your server.
Surprisingly (to some) these are easier that self-hosting email. So this is a great article than I plan to add it to my-digital-self-reliance playbook.
I also agree with the motivations and have a whole list of others. We are becoming the slaves of Big Tech. Only go there willingly, don't let the hard choice of saying "no" make the decision for you.
Are there any (Self-Hosted?) alternatives nowadays?
1: https://mailinabox.email 2: https://aws.amazon.com/workmail/ 3: https://www.microsoft.com/en-us/microsoft-365/exchange/excha...
If you let S&M loose from your main domain with spam then you will quickly vanish. If you have an IP that belongs to a consumer ISP then you will need to do some initial "polishing", which will probably start at Spamhaus. In my experience, if you keep Spamhaus on side then most email systems will give you a fair look. Get the basics right - (E)HELO => DNS (A and PTR) and SPF with -all on the end. Ideally your SPF should be "mx -all" which means you send and receive email on one small set of gateway systems, with no includes which implies marketing mail. That's why you should spam errrr market from another domain or a sub domain. Use Mail Pig or whatever to do your dirty work - that's what they do.
Depending on where in the world you are, then your IP will accrue some reputation - good or bad. Sorry can't help there but a VPN or a remote VPS might sort that out.
There's a bit more to it than the above but that gets you started. Anti spam systems are not psychotic and don't hate self hosted setups. Even the big jobbies have to still give the benefit of the doubt to a sender but they have a huge rule set of heuristics and a twinge here and a poke there might cause you to vanish. It's all about scores and scoring points. Avoid scoring points and you will be fine.
My MTA of choice these days is Exim with rspamd. rspamd has a great symbols to points system where a symbol might be RDNS_IS_BOLLOCKS and the score for that symbol might be +1.5. You can fiddle with scores in the web interface so you can override the defaults quickly and easily. You might decide that SENDER_DOMAIN_DOT_RU gets +10. I've made up my example symbols for this post but you can make up your own like these in the system quite easily.
Anyway. There are rules for running email systems and if you follow them, you will probably be fine.
Ahem. However, I now have accumulated more downtime than I ever did hosting things myself, except for that time centurylink through apparent sheer incompetence nuked my DNS reverse mappings for a month.
I have to admit I was flying under the radar, and my current provider is not. So I will happily continue to pay.
[1] No names, they're great, even if I bitch here.
That said, there's no better option so I've been running my own mailserver for 10 years now. It's even easier when it's only for you and you don't have to implement oh-so-hackable webmail interfaces.
I self host for the former and send through a smart host for the latter. I can’t begin to enumerate how much identity I have accumulated over the last 30 years. I must be known by hundreds of ID tokens (email addresses) and yet I have only ever sent from a handful.
Blessed is the inbound SMTP. Outbound* is a cruel mistress.
*to gmail et al
It's never been easier to self host your email with projects like the following around:
- https://github.com/albertito/chasquid
- https://github.com/haraka/haraka
- https://github.com/mail-in-a-box/mailinabox
- https://github.com/Mailu/Mailu
Of course the usual dovecot + postfix setup is great for learning even if a bit complicated.
Nowadays I use ProtonMail and I get most of the features that GMail gave me, with the added benefit of not managing the blacklist situations.
I just had an invoice from someone using bill.com blocked by spamassassin, since I set 4.7 as my spam level (it was 4.8)
2.5 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From 1.3 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of words 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain
Usually I look in my spam folder once in a while, but I was busy in the past month and didn't know he had sent it.
Anyhow it works well enough for me. I set up SPF in DNS a few years ago. Have not had to do DKIM or DMARC yet - if I need to, or if I have the time, I'll probably put them in.
It's nice to have full control of your mail, DNS etc., but I have been doing it so long it's second nature for me.
Sounds great! Can't argue with that. My feeling is that the real problem isn't a company or companies offering computing services. That has always happened and will always happen. I think the real problem people aren't grappling with is vendor lock-in. Most of the catastrophic anecdotes I read on here and elsewhere are about people who put all their eggs into one basket and did not have any kind of disaster recovery plan. When their provider service went down or even went away due to a merger or whatever, they were left with nothing. And that's really a different problem.
To think even RedHat hasn't self-hosted their email for ages, definitely back to pre-IBM days.
Makes me wonder which major distros are still dogfooding the mail server software they ship.
- docker-mailserver is excellent for a number of reasons: outstanding documentation, sensible defaults, long-term maintenance is a priority, timely updates, small footprint, etc.
- I've only ever had one issue sending an email to a medical provider that was using a blocklist maintained by Proofpoint. Getting off of Proofpoint's blocklist took quite a bit of effort and escalation (their online unblock request process is a joke).
- Migrate uses of your email account over in phases and take your time to build up confidence. I can not stress this enough. I'm still not fully cut over yet. I did marketing, mailing lists, subscriptions, etc first. A year later, I'm still gradually cutting over medical/financial/important accounts. Only after I'm 100% migrated over will I try to get my family to switch over.
- Surprisingly, I have received zero spam. I use the me+netflix@mydomain.com pattern religiously
- I have a cheap $5/month VPS that fronts the public facing bits and run all traffic through a wireguard tunnel to a server at home where docker-mailserver runs. This was fun to set up technically, but in retrospect, creates more points of failure than it is worth. I will be changing the architecure to host everything on the VPS and get a slightly larger instance.
- I document every incident, downtime root cause, etc. It helps immensely when a failure that occurred 6 months ago happens again and I don't have to spin my wheels figuring out what the magic incantation to fix the issue is.
- Do it if you enjoy this sort of thing. I knew close to nothing about email infra prior. If you want an "appliance" like experience with zero maintenance, stick with your current solution.
My one major unresolved issue with going this route is the SPOF, me. My email solution and everyone dependent on it fails if I get run over by a bus. I don't know what the solution to this is, but it has gotten me thinking about decentralized solutions geared towards self-hosters by self-hosters. The goal would be to capture the essential requirements that self-hosting email (or any other service, really) fulfills, and building a solution that scales beyond any dependence on me. This is fun to think about - the decentralization, performance, scalability, privacy, and reliability aspects are well suited to lots of tech that are relevant today.
I find the emails that get through from one email source in one country to another to be quite interesting. Are AV/AS systems blocking the email or something else?
One issue after I moved boxes & IPs at OVH is that Microsoft refused to accept mail from my new IP no matter what I tried. Everyone else is fine. So I have to relay live/hotmail destinations via another jump on a VPS I have.
2-3 years, so far so good, minimal maintenance.
>> "While I’m not going into specifics regarding postfix, dovecot, etc. it’s important to mention a few architectual details."
I trust the teams at large email providers far more than myself to secure it. Not worth the risk, IMHO.