I love this idea (if only for the motivation behind it).
How would your "normal" mail client filter out the messages that are for this app? People have lots of different mail clients which they might not be able (or willing) to configure. Or would you expect a dedicated address.
The app would run on the user's local PC or phone, and it would access data directly from their mailbox (and create a local DB from that data). So no backend server to worry about.
Without a third-party managing associations, how do you maintain that "friendship" is consensual? In other words, how do you prevent "I have your email address" from meaning "you're my friend now."
Almost all email clients have some sort of filtering. And the email messages would all have a keyword in the subject line, so filtering could be accomplished by that (or via an x-header). The app itself could automatically create the filters for various web hosted email systems too.
