I have a similar pipeline (bank and credit cards included) although I run my own SMTP server to capture the transactions as they're forwarded from my main email account. It's a pain to setup the first time but not hard to maintain if you lock it down.

It started out fairly basic (just CSV files dumped somewhere monthly) but the more history is there the more I find to do with it. I recently rewrote the whole thing and added auto-reconcile support, so it can tell if my record in Ledger is wrong and point out the exact discrepancies. I have a tendency to be lazy about recording things so making the machine check everything is nice.

It sucks that there aren't any good APIs for this sort of thing. But until that happens I'm stuck running Postfix and scraping HTML out of mbox files.