Check out Chase's privacy policy as an example:
https://www.chase.com/digital/resources/privacy-security/pri...
A number of information sharing activities cannot be limited. This is typical of any bank or financial institution. Your bank has its own vendors, many of them are themselves SaaS and cloud hosted!
Even large, sophisticated banks can be hacked:
https://www.nytimes.com/2019/07/30/business/bank-hacks-capit...
My point isn't to say "Why care at all? Just open the floodgates!" Instead, my point here is that trust and security in our society is only as good as the people and institutions that back them up. We don't use bank vault doors for our front doors just because we have the knowledge that anyone with simple tools can defeat a home lock.
Therefore, I think that the choice of more inconvenient solutions made just to avoid some nebulous what-if scenarios involving privacy is often (but not always) the wrong way to go.