[1] - https://en.wikipedia.org/wiki/Hashcash
[2] - http://hashcash.org/papers/pvp.pdf
[3] - https://moderncrypto.org/mail-archive/messaging/2014/000782....
The anti-spam system is basically what you said: you spend a small amount of bitcoin, and the server grants you some large number of "tokens". These tokens can be used to perform actions which use resources on the server, either storing messages, or getting messages. The exact price is configurable, and currently set low [1].
There is a second mechanism that I use, which is intended to lower costs for the average user, but still prevent spam. Essentially, in each bitcoin transaction, a small "transaction fee" is required, which goes to the miner. It is possible to prove, via a cryptographic signature, that you were the originator of a transaction, and therefore the person who spent that transaction fee. The bitnet system grants tokens to people who can prove they spent money on transaction fees [2]. The idea being, a typical bitcoin user will accumulate transaction fees anyways, but a spammer will have to go out of his way to send bulk messages.
[1] https://github.com/ortutay/bitnet/blob/master/bitnet.go#L50 [2] https://github.com/ortutay/bitnet/blob/master/bitnet.go#L38
The idea seems to be that you provably "burn" a small amount of Bitcoin to get an identity. An innocent person can then carry on using that identity forever without doing any more computation. Meanwhile a spammer will ruin that identity's reputation almost immediately and then have to pay again to get another one.
There's also botnets, whose resources hijackers are happy to exploit.
OpenBazzar a bit-torrent like p2p version of ebay uses proof of burn to make bad behavior expensive.
All of my mails are newsletters containing 10-30 links, and more than once I've found the mere inclusion of a single link to a certain domain can get something into spam versus a version without that link, often with no clear reason why (domains that are particularly new are one marker, though). Or.. how about using a Unicode 'tick' symbol in a mail? That can get a reputable sender into Spam versus a version without the same single character (all double tested against a clean, new Gmail account) :-) Or how about if you have a link title that includes both ALL CAPS words and ! anywhere? Your risk goes up a good bit, but just go with one of them, you're fine..
I now have a playbook based around numerous findings like this, some based on gut feelings looking at the results and some truly proven, and even with my solid reputation as a sender, I'm having to negotiate a lot content-wise each week. But do I like it? Yeah, in a way, because it's also what stops everyone else being a success at it.. Gmail sets the bar high! :-)
(Oh, a bonus one.. include a graphic over a certain size? Your chance of ending up in the Promotions folder just leapt up. Remove it, you're good. It doesn't seem to be swayed much by actual content. So I've stopped using images where at all possible now and open rates stay up because of it.)
I think there could be multiple, relatively easy methods to avoid encrypted spam.
Someone here already suggested the first email being a "poke". And only if you send a poke back, would that user be allowed to send you an e-mail.
The user could also have some description about him, from his profile, appear when you hover over his profile image or whatever. If you receive an e-mail say from a company you're expecting to receive email from, then you could poke back, so they can send you that email. I mean there should be ways to make it easy for people to know who's a total stranger that could be a spammer, or someone trying to reach out to them for good reasons.
Then you could also have the emails under different labels by default. All the trusted e-mails would come to the regular Inbox, while the rest will go under a different label.
As you said, the email provider could also see the user's reputation over time, and if he's a spammer or not.
And these are just some easy solutions we can come up with almost immediately. I'm sure there can be others with a little bit more thought put into it. I certainly don't see encrypted email as some kind of "doomsday scenario" like the author predicts in the post.
There are at least three major downsides:
1) You still leak lots of metadata and the full data of the poke including most obviously the subject line.
2) Do users understand that their spangly new "encrypted mail" actually fails to protect a lot of important data? What if they (gasp) came to rely on it? I'd want to see usability studies showing a clear understanding of what is protected and what isn't.
3) You break other features that rely on the server being able to see content, like search, and the ads that pay for all of this.
It seems to work reasonably well, although there are some interesting ways you can game it. One I learnt from the Internet marketing world was some list builders (using legit methods, but perhaps promoting things that often get caught by spam filters) hire people or implement techniques to encourage new list signups to reply to mails sent from the same address as the list by asking them questions, etc.
I realized I should add a note, however, that everything I've said only applies to bulk e-mail (and sent through systems with a reputation for such) and not transactional or manual e-mail which suffers from fewer oddities for obvious reasons.
Trying to game the system in web search is a huge thing, cause there's a big reward. But G fights it pretty well.
Not such a bad idea G publicly ranks mail senders.
I know that's probably not why it's private, but it got me thinking.
So basically, I can't send email from home? This is… unfortunate. If we want freedom, we need decentralization, and this kills it.
One (open, centralized) system for global comms. Another (closed, decentralized) for secure comms. Maybe even more than one "another", if contexts require different audiences.
And there's the case of looking for work. I'm somewhat proud of showing my personalized domain name, but if using something other than a huge webmail can cause it to fall into a spam folder… Fortunately that has yet to happen, but this is one of the reason I hesitate to switch from remote SSH to a physical server at home.
Email is supposed to go from the sender's machine to the receiver's machine. That's how it should work by default, that's how TLS connections makes the communication vaguely secure, and that's how it makes it difficult for powerful third parties to have a peek at everyone's communications.
As far as I know, Gmail accounts are only a subpoena away from the US government. But a sheeva plug (or R-Pi) hosted at my home? They need a warrant. Even for countries that don't need warrants, wire-tapping everyone is expensive: it must be done one home at a time.
Now maybe the botnet situation is so bad that it is worth sacrificing our ability to send e-mail. Still, this strikes me as the wrong solution. Blocking outgoing 25 by default is fine, but we need to be able to lift the restriction if we want.
One issue I could see though is the initial email would essentially devolve into a "poke". Nobody would bother writing anything in it, which would mean the spam filters would have nothing to filter on.
that is a good thing: if the first message contains something else that is not just "poke", it's spam.
It would provide protection against passive snooping (NSA/GCHQ) even if it wouldn't prevent active attacks.
Edit: Typo
I've been working on custom software to improve the spam filtering on my mail server for the last year (side project). It currently works by letting hosted users forward spam messages to a flytrap account, and then the daemon runs, reads the forwarded message, tracks down the original in the user's mail directory, does a whois on the origin in the mail headers, consults its logs, and then adds a temporary network-wide blackhole to iptables.
Originally it was intended to work alongside SpamAssassin and SQLGrey and all that, but last night I started considering replacing SpamAssassin altogether. I love SA, but the spammers are beating it regularly now. My TODO notes in the code actually say, "reputation tracking for embedded URLs, domains, ccTLDs and gTLDs, sender addresses, and content keywords." I wrote the first bits of code for reputation tracking this morning.
It's not much of a step for the software really, because it already uses embedded URLs in a message as part of the profile "fingerprint" for finding the original message from a forwarded version.
But I'm a bit chuffed to hear that I'm on the right track, considering how effective Gmail's tactics have been. :-)
Small service providers have it really tough right now. Users don't tolerate any spam at all. A few years ago, the state of the art for small independent services was SpamAssassin + SQLGrey (or other greylisting) plus a few other tricks; that's not sufficient anymore, and most of us smallfry lack the resources to come up with something much better.
After just 6 weeks in production, the software already has 20+million IPs blocked at any given time.
Beyond that also one of the things SA doesn't do well is actually rejecting hard on sensible blacklists like the CBL. We worked hard to make everything heuristic based but it wasn't always the right choice. Some things need to be black and white. There's some code on SA for short circuiting now but it's not really the best solution. In my own spam filtering I have a bunch of hard rejects and they work really well.
Anyway, check out Haraka or Qpsmtpd for solid anti spam mail serving solutions. They work really well.
There are some challenges though. Probably the biggest one is that it's designed for a specific mail server setup: postfix + dovecot (configured for maildir) + fail2ban + php (the code is in php, because it was convenient) + mysql. I don't know yet how portable it will be.
If you'd like to try it anyway, let me know and I'll post what I've got to GitHub in the next few days.
edit: alternatively, I've been more seriously considering making the current network ban list available as an RBL. Since I already have DNS servers, it would be pretty trivial to do.
This wouldn't work, because a miner can easily pay himself any amount of bitcoins that he has saved up in fees, and include this transaction in his own block (not broadcasting it). Thus he can basically create these "deposits" for free, and sell them for a profit.
That's the thing: whatever you try as a counter-measure, you always come back to money: in the above scenario, money would replace "deposits" because "deposits" would just be sold on the open market for money. Proof-of-work becomes money: if something important requires proof-of-work, you can be sure that a web app would surface that performs proof-of-work in exchange for money.
It always comes back to money, because whatever restriction you put on something, whether it be "pay fee to Bitcoin miners", "Solve proof-of-work puzzle", or something else entirely, these things will always end up being sold for money in an efficient market, because of the increased efficiency of division of labor: why should I use my inefficient smartphone to calculate proof-of-work, when I can pay a service with custom ASICs to do the job for me at a fraction of the cost?
As far as I can see, the only alternative that can work besides money is something that cannot be sold for money. And I can't come up with anything that fits this requirement.
Of course spammers can buy their services but when the price for normal sending is effectively free for normal users you can jack up the price for spammers to make it too expensive for them.
A nice benefit is that it forces sites to use something other than email, such as RSS, since they can't afford to send newsletters anymore.
> So I think we need totally new approaches. The first idea people have is to make sending email cost money, but that sucks for several reasons; most obviously - free global communication is IMHO one of humanities greatest achievements, right up there with putting a man on the moon. Someone from rural China can send me a message within seconds, for free, and I can reply, for free! Think about that for a second.
Obviously, both of us need computers, email accounts, network access, etc but there's no per region metering or anything. The cost of sending an email to someone sitting 10 feet from me or 10,000 miles from me is exactly the same. Mike's right, this is revolutionary.
Someone from rural China can send me a message within seconds,
for free, and I can reply, for free!
This offer is only available for you alone and in person. Are you interested? Because I'd really like to know how your pedantic definition of "free" works out for you when it's obvious that things that require infrastructure (such as online communications, or traveling to a different country)... require infrastructure.
(TLDR: If you don't find it amazing that with a very small indirect investment we can actually communicate with people anywhere in the world for free, you and I need to have a long chat about being spoiled.)
Email currently uses a Receiver Stores model. SMTP servers can relay messages, but in almost all cases the message is transmitted directly from the originator's network to the recipient's network. The storage of the message only effectively changes _ownership_ once, even if the message headers say it was forwarded many times.
That makes email a Receiver Stores model: the recipient's network is expected to accept the message at any time and then hold it until the recipient comes to look at it.
Some of the bitcoin messaging protocols propose a Sender Stores model. That is, the message may be transmitted any number of times but the recipient's network is not responsible for long-term storage. The sender's network must be able to provide the message at any time up to the point when the recipient actually looks at the message.
There are some obvious restrictions such as requiring that the message be encrypted with a Diffie-Helman key (negotiated when the message is first transmitted to the receiver's network) to reduce the feasibility of de-duplicating millions of messages. And in order to prevent revealing exactly when the recipient reads the message, the recipient's network doesn't ack the message for a while.
Ultimately all of this is just designed to make bulk email (slightly) more expensive. Spammers run on very, very thin margins. But it doesn't do anything to solve the problem of account termination or blacklisting.
Since early recipients of the spam will likely report it, it's fairly likely that subsequent retrieval attempts will find a downed (or sanitized) host, no longer delivering spam.
This will reduce the amount of spam actually delivered, and the spammer's production / revenue margins.
See http://cr.yp.to/im2000.html and http://en.wikipedia.org/wiki/Internet_Mail_2000
End-to-end encryption is needed to increase the load on a spammer as much as possible. Even if the spammer tries to re-generate the message "at retrieval time" the receiver should request retrieval several times (to obfuscate when the message is actually read) and the message should use multiple iterations of a cipher (and possibly HMAC) after an initial DH negotiation, or any other means to increase the cost for a spammer _and_ tie a message to a unique sender for reputation-tracking purposes.
Why is the cost of end-to-end crypto never taken into account?
I just can't believe that we have reached a point where it is possible to cheaply mass mail the way spammers do if you need to encrypt each email for each recipient. That alone should be disuasive enough, at least that's always what I thought. If I'm right, all the discussion about the need for client to extract features from emails and send them to a necessarily trusted centralized third party is useless. But I may be missing something, where am I wrong?
But let's make the crazy assumptions that we are effectively in a world where end-to-end crypto is massively used, virtually by everyone.
I guess it would be stupid to assume that the message are encrypted but not signed.
This means that it would be easy to have a list of identities (i.e., keys) which are sending spams, for instance using a web of trust, without users having to disclose any other informations that "I trust these identities, not these ones" (which could be as easy to do as clicking "spam" or "not spam" for the users).
Now that means that the spammers not only have to encrypt every single email for every single recipient but also to generate new key-pairs for almost each encryption.
Of course it is also very easy to mark as spam any email signed with a key that is considered too small (i.e., too quick to generate).
Now if you tell be that still won't do it without a "centralized spam system with global knowledge", I have to seriously rethink a lot of my assumptions about the cost of some computations.
The first incident was that Gmail flagged an important email from my landlord as spam because it contained a forwarded message written in Danish, which the filter deemed to be a language I don't normally correspond in (it is nice, to be fair, that the filter actually tells me why it flagged the message). True enough. But I do live in Denmark, and in fact a mail containing Danish is a very good signal, for me, that it should not be spam-filtered.
I've moved to hosting my own mail as a result, and it's been going well so far. I use a fairly conservative host-based filtering approach. Just blocking hosts whose DNS doesn't match their rDNS rejects >70% of spam attempts, and adding Spamhaus's DNSBL brings that up to >95%. As far as I can tell from perusing the logs, it's quite conservative, and they're all true positives. And at least it rejects (if it's going to) in the SMTP session, so the sender will get a bounce rather than get silently filed into a spam folder, like Gmail does.
I do still get some spam, almost all of it from legitimate free-mail hosts who I can't feasibly filter by host (mostly Yahoo and Gmail). But it's fairly infrequent.
(in mailing lists, and yes, I checked, it seemed to be a legitimate email, so I let the sender know, but never heard back)
Could anyone comment how spammers make money actually?
And cranking up the proof of work isn't going to do anything to prevent it. The only thing that prevents Bitmessage from becoming a cesspool is its obscurity.
If you didn't obtain a fresh piece of this script, but instead either reused, or tried to guess the token, then your signup still succeeds, but is marked as bad. Then, in an undetermined amount of time, a wave of bans would occur on all accounts that got marked as bad.
This prevents signups via scripts, but does so via making signup scripts untrustworthy, and so no one would be willing to put money in for they cannot be sure that it actually works.
There is idea that this somehow blocks access to email for people who have a hard time paying for things on the internet (for whatever reason), but it is misguided: everybody who has access to the internet pays for it. ISPs could easily give every subscribed 10000 free emails every month.
Texting costs money and yet people do it.
What am I missing?
The specs and the software for a start.
Then there's the fact that it's not the spammers that will end up paying, but the people running the systems that are compromised and abused to send spam, be they shared hosting servers or home computers.
Then there's the network effect. I'm not going to feel good telling my friends and family that it will now cost them money to email me. Especially when they can just contact me using Facebook instead for free and without having to set anything new up. Especially when the email service they're already using probably wont even support this new fangled paid-email system.
It would be a massive task to add this functionality to email, and it wouldn't stop the spam, so it's not worth it.
Alice wants to receive mail from Bob. Alice generates a public/private key pair and gives the public half to Bob. When Bob wants to send mail to Alice, Bob uses the public key Alice gave him. If Alice receives spam, she marks the public key it was encrypted with as "fuck it, the spammers got it" and never receives mail with that key again. Then she notifies Bob that the key he had has been compromised and sends him a new one. Alice could then, after Bob has lost her key to spammers one too many times, simply decide not to talk to someone like him.
This would give mailing list operators a large incentive never to share your email with anyone, otherwise you could just block them forever.
On the flip side, if the mailing list is really important to you, the operator could reject your new key and tell you you'll either receive their spam or you won't be part of the mailing list. Though I don't see why someone would do that in favour of just including ads in the mails themselves.
Alice gives her key to Bob, in the expectation that Bob would not be sending her spam. Bob then sends both spam, as well as legit mail that alice did want. Assuming that alice does not want to stop receiving the legit mail, but want to stop the spam, how does she do it in this scenario?
If alice blacklist the key for bob, but sends a new one, the situation didn't improve. If she doesn't send a new one, she stops receiving legit mail (that she wants, and cannot go without).
How does that work? Don't SIM cards cost more than 10 cents?
In a system like that, maybe you could send your encrypted message along with some encrypted keywords that you consider to be spammy to some centralised service. That would, at least, avoid some of the client-side-filtering-is-too-hard problem.
As far as reputation, this might be one of the rare times where a Web of Trust seems like a good idea. Generating lots of false positives and negatives would be a lot less powerful if the value of those reports was filtered by how much you trust the account that made them. With email you already have an implicit source of trust, in that anyone you mutually email with is unlikely to be a spammer.
Seems like a really interesting problem space to be involved in.
I switched to gmail because my mail with every other provider and client was choked with phishing messages from major banks. So much work has been done on preventing origin spoofing in 2014 that accepting phony mail from chase.com is a sign of gross incompetence.