Care to name and shame that bank? I would like to avoid any bank that doesn't use a website.

Really? And you are okay with losing access to your money if your phone is lost/stolen?

If not, you should seriously consider switching banks (while you can). I suspect that such banks do not take security seriously: Giving control over your phone to Apple/Google is not security.