I've been a happy customer, even though lately I flirted with going back to GSuite for my personal email, but after a trial realized that Gmail does many things well, except for being a good email service. So I went back to FastMail and renewed for another 2 years.
Seeing this new protocol is exciting, because JMAP is being standardized at IETF. A breath of fresh air to see a new standard being developed.
Also from what I understand JMAP should be friendly for mobile usage. They kept notifications out of it, you're supposed to implement notifications using whatever the mobile platform provides. Interacting via JMAP is via plain HTTP requests, which is super cool.
I can totally see myself implementing a simple email client for automating online services. For example if you implement a commenting system for a website, you might want to do replies by email. That would be a cool project for me to try out.
I wonder if FastMail exposes JMAP publicly yet. Haven't seen any mentions in their admin or docs thus far.
The worst part is I think Fastmail is aware of it and just don't care (believe that's why they mark their emails with a green tick and text). I understand that email has never been really authenticated, but this just throws any trust I had in Fastmail out the window.
I will be evaluating other mail hosts at the end of my subscription.
SPF has nothing to do with the From header. And the DKIM signature does not have to match the sender’s domain, the signature can be that of any domain. This means that for practical purposes, anybody can send spoofed emails. That an email is signed with DKIM, that doesn’t mean much and it is meant to build a web of trust between servers, but otherwise it is useless for the users themselves.
They wrote a blog post about how SPF/DKIM work: https://fastmail.blog/2016/12/24/spf-dkim-dmarc/
If you want to let people know which emails are from you, the From address is very weak. This is because the From/To headers tell you nothing about the source and the destination of the message, according to the email standard. Read that blog post for details.
You need a proper signature via PGP or S/MIME if you want to ensure that the receiver knows the message is from you. And unfortunately this requires education and email clients with support for such signatures (most desktop clients do), but that’s email for you.
https://www.fastmail.com/help/technical/senderauthentication...
I reported the same problem to posteo.de in March 2016 and still have not received a satisfactory answer, though it seems they have some counter-measures in their webmailer nowadays. The fun part was that as a "no logs" privacy-oriented provider, they were not even able to track who sent them a complaint from their own support address ¯\\_(ツ)_/¯
As a comparison: at disroot.org I found the same problem, and it took them a few hours to repair their postfix configs.
But anyone can set up their own postfix/qmail/sendmail server and put anything they want as the From.
Or am I misunderstanding the issue here?
The problem is if any email service did this you'd start trusting the "from" field and that is wrong. Do not trust the from field. It's as simple as that.
This.
I remember back when Gmail was new and hot. It was unlike any other email service out there, and ridiculing people for using inferior email-solutions could to a certain extent be justified.
While other webmails were slow, had constantly reloading pages and what not, Gmail was fast. It was amazingly fast. Gone where the 5 minutes making the webmail work for you. You just sent the email and you were done. Just like that. Back then, this was unheard of.
These days though? Everyone is still pretending like GMail is the only game in town, when I think they have one of the worst webmail-interfaces out there. And it's slow. Oh god it is slow. And god forbid you try to load it in a browser not Chrome, because then it just grinds to a complete halt.
So yeah. Happy Firefox-using FastMail-customer here. You couldn't get me back to GMail even if you paid me.
(I got my first gmail account in 2004.)
Yeah, the UI wasn't bad, but there was less email back then and I didn't (still don't) really mind being served HTML repeatedly instead of some "AJAX" application, which is what I remember the technique being called. (It used to be the latency to the server was poor and JS hid that; now, on 2-3ms fiber, it seems that the JS actually introduces a lot more latency than it hides.)
I can't even begin to consider using it at the price point I presume I'd be in.
For IMAP folders you can set a default identity and email in a folder stays in that folder without cluttering Archive, like in Gmail. So in effect you can do much with a single account.
I read the FM blogs and I remember them mentioning that low battery consumption was one of the design goals.
The JMAP site [0] has:
> JMAP is designed to make efficient use of limited network resources. Multiple API calls may be batched in a single request to the server, reducing round trips and improving battery life on mobile devices.
Sounds actually not cool. Is there really no standardized way to do notifications? The world consists not only of shitty mobile walled gardens which insist on or strongly favor their proprietary implementations.
Gmail/Gsuite seem to be good email services from my perspective as lay user and occasional admin. Can you expand on why you think they are not good email services?
- constantly changing things around. UI gets less intuitive for every release.
- slower for every release
- violates IMAP standard by re-using IDs across tags/IMAP folders, risking actual data loss. Example: if you in a real email-client try to delete a email from a single folder, you will also delete this email from all other folders it has been "tagged" in.
- similar IMAP issues with sent emails. Hard to track sent emails from email-client, unless you allow the email-client to explicitly save a copy to sent-emails folder. But then you suddenly have duplicate emails in the web-UI.
- Makes an open standard (internet email) proprietary.
- Hell to integrate with: See above.
I'm sure I could go on, but really. If you still consider GMail best in class, the only possible explanation is that you haven't seen anything else.
Describing this as REST is really strange. Defining your own operations over an HTTP POST is what SOAP and other RPC style web services do and specifically what REST isn't. But I guess that a lack of a standard behind REST ended up with the term being used for everything.
It involves two endpoints exchanging the state of a shared resource. It needs to be compliant with the constraints of that style.
People think that REST must be over HTTP, but it can be over any protocol. The essence is that it is a style of systems design, so JMAP can be considered RESTful as described in the link above.
REST is one of the real patterns in software architecture, a set of constraints, not a set of structural elements.
The fact that this is a discussion though makes my point. By REST being only an achitecture style with very loose definitions makes it arguably fit for all different kinds of APIs which in turn has made the term useless over time. Maybe we could use a set of technical standards (like SOAP) for different common API implementation solutions within the bigger REST idea. Discussions like REST vs SOAP are like comparing OO and Haskell, one is a concept/pattern, another is a specific technology.
Also i'd classify a REST api as anything that does http requests and consumes proper JSON. As long as this is true , the rest is squabbling :)
REST doesn't need to use any HTTP verbs other than GET and POST.
Maybe I'm naive in thinking it was possible but we could have avoided this whole "make an account on X messenger so we can talk", if someone had just jammed XMPP and SMTP together.
Every few years I'm tempted to try and write a SMTP server (MX) myself but then realize that postfix has been around so long and is the go to choice for a reason. I don't have 20 years of figuring out how X mail server interprets SMTP to be as reliable as postfix. I settled for trying to wrap it[0].
The real hole is "instant messaging". Somehow every few years we got from ICQ to AIM to Paltalk to Skype to Facebook Messenger to Slack to ... (at least Altassian had the decency to take HipChat behind the woodshed and shoot it)
The strange thing is that these services dry up, blow away, get replaced, but they don't seem to improve on what came before.
There is a standard, XMPP, but the only people who care about it are firefighters, cops, and soldiers. For all the anger at internet giants these days, I can't see for the life of me why people aren't pushing for open instant messenging.
The second point is especially big. People expect to be pointed to a singular app/site, not “choose whichever client you like”. Your average Joe just doesn’t conceptualize services as being separate from clients in the first place, and even if they did having to choose a client is a dead end.
There would obviously be a lot to work out, but if something like this was in the SMTP standard (or at least introduced), I think eventually email providers would race to differentiate themselves with support for it, and we might not be where we are today. In my view there's no reason to even require the SMTP servers to serve chat traffic -- as long as they could hand off reasonably, and everything spoke the language as specified in the spec (or at least came close).
Xmpp is/supports federated/ion (user1@server1.com can message user2@server2.com).
But the major players (by numbers) wanted silos: Google (talk) and fasebook (first gen of messenger).
They basically went lol, screw users (arguably because: spam. But hello, Gmail? And today fb spam...).
So thank Google and Facebook for deliberately gimping it so you can't fb message user@gmail.com or gtalk first name.lastname123@facebook.com.
> make an account
You are required to make an account somewhere so your account can be banned if you spam, basically.
I thought I searched far and wide for other mail servers, yet only came up with postfix, iredmail and cyrus, I clearly didn't look hard enough
Umbrella's for example see continuous evolution in means of making better fully collapsible versions, making the deployment more automated, reducing weight, etc.
From Wikipedia, illustrating both that there's continued substantial effort in coming up with new ideas, and at the same time that it is hard to come up with something genuinely new:
> Umbrellas continue to be actively developed. In the US, so many umbrella-related patents are being filed that the U.S. Patent Office employs four full-time examiners to assess them. As of 2008, the office registered 3000 active patents on umbrella-related inventions. Nonetheless, Totes, the largest American umbrella producer, has stopped accepting unsolicited proposals. Its director of umbrella development was reported as saying that while umbrellas are so ordinary that everyone thinks about them, "it's difficult to come up with an umbrella idea that hasn’t already been done."[36]
I have view on the spam thing. The whole "your idea will not work because" meme is hugely destructive of innovation in email. It sucks energy and mindshare. It's classic old timer put down.
What would (imnsho opinion) have fixed spam is sender pays. I've debated this with a lot of people. We're 50/50 on it. Fifty agree with me, fifty million don't.
Jmap is utf8 clean btw.
(I used to do email for a living in the eighties when life was simple and bang!chains!worked)
I propose a scheme where the sender pays e.g. 5 ct per e-mail (low enough that it does not matter for legitimate use, but high enough to make spam unprofitable). BUT with the following twist: The receiver can generate API tokens that allow free e-mail delivery.
So when I sign up for e.g. Twitter, I can give them an API token for my mail address so they can send notification mails to me for free. If they decide to start spamming me, or if they decide to sell my token to a spammer, I can just revoke the token and the spam flood stops instantly.
I have not found any flaws in this idea yet. RFC!
Natural solution is a pricing list, but then USA spammers could route via cheapest geographic servers.
How do you get paid? Banks will put a transaction cost making sending email cost stupid money. So, bitcoin? But the built in energy costs for processing a transaction will force a floor on the transaction pricing that's too high?
If a token leaks you'll have to fiddle about to allow a sender's emails to get through ; I guess you could automate that if your MUA had credentials to inform genuine senders of API key updates.
Then you funnel all that junk into a special folder (black hole) that you can either ignore or check if you are expecting a legit request for your attention.
The more I think about it, the more I love this idea. All email that shows up in your inbox is there because you have to explicitly allowed it.
The one controversial part of this is deciding who is permitted to redeem the bond, i.e. who is the central arbiter of what is and isn't spam? Well, fortunately we already have an entity like that, de facto, since we have the Big 4 email providers ("Gmail, Hotmail, Yahoo, and AOL", or any other similar group that all mail servers must, in practice, comply with the demands of).
In the system I'm proposing, if any of these big providers detect a domain being involved in spam (or N out of M providers, to reduce the risk of false positives), they could redeem the bond. Of course, the bond would be structured such that the only recipient was a non-profit, like the IETF, or possibly an entity like ICANN. That way there is no financial incentive to make false positive claims.
As for how this system would be introduced, that's the easy part. All pre-existing domains would be grandfathered in, so that no current email users would be negatively affected. (Indeed, there is a moral hazard that some email providers would support this system precisely because it only affects new entrants to the market). There would simply be a flag day, after which if you register a new domain, and want to send emails from it, your domain registrar would have a check box saying "Yes, I am happy to be charged an extra $10 and have put it in a bond so that I can send email to the Big 4 providers". After sending a certain amount of legitimate email, a Big 4 provider could then send a transaction which makes the locked funds return to the original issuer of the bond.
Finally, to decide how big the bond has to be, we can look at data like this:
https://krebsonsecurity.com/2018/06/bad-men-at-work-please-d...
If spammers are going to extra effort to save a few dollars on their registration costs, then their margins must be pretty tight. It also means that domain reputation systems are working, since spammers have to keep bulk buying new domains:
http://conferences.sigcomm.org/imc/2013/papers/imc247-haoA.p...
The obvious downside is that since the system is fundamentally just cryptography it doesn't discriminate between spam and legitimate bulk mail (password resets, newsletters, mailing lists, etc.)
I don't really see the payment as a big problem. This has been discussed previously, mostly initiated by the big email services
The sender pays the reader. If the reader is the one deriving the value from this communication, they would pay the sender out of band as compensation.
Along with the billions of spam SMS that are also supposed to be sender-pays, sadly
Alternate but similar - sender holds the email until the receiver fetches: https://cr.yp.to/im2000.html
I'm not sure it actually helps that much to cut down spam but anything is worth a try at this point.
Email is designed around asynchronous clients, e.g. the ability to write everything offline, connect for just long enough to queue them all up in some server, then disconnect while that server passes them along. By the time the receiver (server or client) knows that they've been sent a message, the sender may be long gone.
Hashcash only works if:
- It can't be bypassed. Without a mechanism to tell senders how much to include, receivers must set their price near zero to avoid discarding legitimate messages which guessed the wrong amount. Keeping messages without any/enough hashcash would defeat the purpose of the thing.
- The sending client does the work. Since servers are always online and reachable via a known address, we could have them negotiate an amount; e.g. the receiving server checks the message for hashcash, returns an error stating that a certain amount is required, the sending server tries again with more hashcash. The problem is, getting the sending server to mine hashcash won't stop spammers: they'll just use other people's servers, like gmail, hotmail, etc.
I've written about this before, but I think that a generic protocol for negotiating hashcash would be really worthwhile. Maybe it could be made to work for email, but even if not there are plenty of synchronous protocols which could use it.
In particular, there's no reason to keep a fixed price; we can figure out a price using the same heuristics as existing spam filters: can we verify the sender, have we seen spam/ham from them before, do they appear on black/whitelists, etc. This way the pressure can be kept on spammers, whilst the majority of normal traffic can go through with little effort. Note that this fixes the mailing list problem too, e.g. if users add the list address to their whitelist, or if they send a message in order to subscribe (hence triggering the "allow this, it's a reply to our message" heuristic).
I also think this would be a nice alternative to API keys, since it would keep things more "open" for experimenting and mashups, whilst giving providers a way to avoid abuse (API keys could still be provided, as a way to significantly lower the amount of hashcash required).
The idea was dead on arrival.
As protocols are meant to connect different implementations and bringing more protocols for the same task quickly hurts the interoperability (before bringing some improvements in the long-term), I am skeptical to the effects those new protocols bring to the ecosystem. I am aware that JMAP was born more out of the RESTful requirements of an HTTP based App, but in general, I am wondering where this road will lead us.
[1]: https://github.com/siacs/Conversations#but-why-do-i-need-a-p...
As for XMPP battery consumption I didn't see major problems, Conversations.im is always <1% (I just checked and it shows 0%). On the other hand Conversations can use push to optimize battery usage.
So it is pretty obvious that the vendor built battery optimization software does more bad than good.
Could JMAP replace SMTP as well as IMAP?
BURL [1] is an SMTP extension that allows to submit a mail that is already stored on an IMAP server. This still requires to make a separate connection to send the mail, but at least it does not require uploading it to both SMTP and IMAP only to store it in the Sent folder.
However, this extension is rarely implemented or deployed, although the RFC itself is already twelve years old. As an example, Dovecot started to support it by acting as a SMTP proxy [2] as of version 2.3 in 2017. I am not aware of any mail clients that support BURL.
Can I know from a capability exchange if the remote server will send emails after adding to the "Outbox" folder?
Background for this is that I implemented a calendar client and a calendar server (with SabreDAV) last year and I'm wondering if I should be adding support for JMAP anytime soon.
> Internet: Do you think JMAP will really take off?
> Me: JMAP is an open, smart, modern, and powerful E-mail protocol, so probably not.
I am not impressed. Folders moved in the webinterface show up unmoved in Thunderbird. And vice versa.
Sometimes it works, sometimes it doesn't. They're investigating the problem at the moment, but the updates they've given me don't give me much hope.
I'm looking at migrating away to another provider that just does plain old imap.
I just do not know if EWS is an "open standard" that could be replicated by a third party server or if it is only "open documentation" and licensed for Exchange only :(
Not sure if this is possible, I don't know if Outlook uses the EWS protocol, OWA or something entirely different :/
WTF? So, webmail was limited by the fact that it was running in a browser and thus had to use HTTP, which has semantics that don't really fit the needs of an email access protocol. Now, we do have stuff like websockets that would make it possible to run a protocol from the webmail client that actually fits the needs, and instead people invent a new protocol that inherently doesn't match the needs of the application?
And could you also explain how constantly making new connections makes things work more reliably over unreliable links? Like, does that allow you to transfer data when the network link is down? Does the fact that inside the TLS/TCP connection data is transferred via HTTP instead of IMAP somehow make the TCP connection work better over lossy links?
It would seem to me like the exact opposite should be the case, if it has any effect at all?
You could have a variant of JMAP with GraphQL syntax and semantics, but there would be a fair bit of mismatch, for their purposes and focuses are quite different: JMAP is concerned with object identity and synchronisation, for what you might call thick clients (and JMAP Mail needs to be broadly IMAP compatible), while GraphQL is UI-focused, for what might reasonably be considered to be thin clients (not that they are logicless, but that they are very much focused on offloading burden to the server). I see no compelling case for a JMAP-like GraphQL thing.
It could still be an interesting task to develop a object synchronisation protocol like the JMAP core on top of GraphQL.
> JMAP is not designed around a persistent network socket, so it’s perfect for webmail clients that connect, do stuff, then disconnect
This would allow implementing JMAP in serverless architecture, e.g. self hosting on AWS lambda. That should be very cheap and significantly lower the barrier to self hosting (not needing to manage a box, just providing aws credentials).
Think of what it takes to truly decentralise email. There are technical hurdles, but also some fundamental ones:
- other providers mark you as spam because you’re unknown
- you need an always on server to actually get your incoming mail
This would at least solve the second problem. If we can develop a product like mailinabox (Which has its flaws but it’s the right idea), but instead of asking for a fresh vm, it just asks for aws credentials, that could be pretty solid.
Hopefully one day we can give the people back their control over their means of communication. This seems like a step in the right direction!
Did I miss something about JMAP?
Mail is decentralized already. Part from relying on DNS.
And you seem very eager to put everything in the amazon basket.
What about for desktop purposes? What would a proper client like Thunderbird or Outlook do?
This isn't true; there's a LOT of industry on Exchange of some sort, and plenty of older institutions running their own email. Especially in IETF land.
And it's a huge risk increase if everyone moves everything to gmail.
Microsoft Exchange used to not bother with actually SMTPing when talking to another exchange server (or itself), I don’t know what it does these days.
