Too "modern" and I assume a site is using excessive amounts of tracking and playing other tricks with A/B testing different marketing tactics for services to try to milk their customers for all their worth.
Too crappy and I start to question if the site is actually secure and if they are putting enough money into IT... or if it's even the real site vs some kind of scam.
Both can be problems that erode trust, but they are different problems.
That said, you probably want to judge your bank's trustworthiness by it's rating with the Federal government, which is public info. Most banks (all but the biggest) have to outsource their online banking software and it's never as slick as the big guys'.
My hypothesis is it's because time and money investment was put into aesthetics at the expense of making the product or service better.
I've even seen it happen in real time with small YouTubers. They get money and start putting it into "production values", then more often than not the quality of their actual substance declines.
Of course, it's not 100%. There are some flashy things that also happen to be decent, and there are nonflashy things that suck. Flashiness is just a general indicator of what the priorities of the maker are.
That, and they haven't yet fallen into the trap of giving their marketing department too much say.
Uglier, less decorated things are expected to function better, with less effort "wasted" on goals unrelated to the problem being solved.
1) "I manage to survive with this crappy of a design, so I must be doing something else right."
2) "I don't track you all that much. I just get out of the way and let you do stuff."
3) "I don't invest as much time in marketing, I invest time in getting stuff done"
I think it's because my brain just associated Linear-looking,Tailwind-ish websites with cryptocurrency, rugpull NFT websites. There were certainly a lot of that few years ago.
This is how I'd explain the impression of trustworthiness. But of course this is only an impression and doesn't tell you anything about how the site actually operates.