Wire fraud is a massive problem. I recently bought a house and I was given very specific instructions about how to verify the wire destination and it was made quite clear that if I was taken in by a fraudster I would never get my money back. Lots of money, like buy a house lots of money. Almost zero chance I would get my money back.
Have you ever worked a retail/food chain job and gotten to see how many CC transactions are invalid/fraudulent resulting in the business losing money? I worked at a pizza place where 5-10% of CC transactions were bogus (college town). Checks were more like 20%.
All financial transactions are trust relationships. Some are higher trust than others. If you could buy groceries at Wal-Mart or a new iPhone from apple.com with bitcoin you would have no concern about that. If you send money to a rando on the internet, well it doesn't matter all that much whether it's crypto or dollars.
Sometimes you will have third party intermediaries, like VISA, who will offer certain guarantees/insurance for a fee. It's as simple as that.
It's silly to compare transactions with known/trusted entities like a physical business or trusted insurance provider/intermediary like VISA with sending your money to a random anonymous bitcoin address. You would not generally be wiring money to a random anonymous bank account either.