IR was exceptionally slow, required line-of-sight and even at the time, felt like a shitty solution. So even though the early implementations of Bluetooth also left a lot to be desired (battery hungry, insecure, and also slow), it was still a massive improvement on what came before.
Wireless USB wasn’t a significant enough improvement to Bluetooth given that BT was already ubiquitous by that point, but also cheap and (by that point) battery efficient now too.
Back when BT was new, I used to get all sorts of random shit pushed onto my phone every Friday night on the drunk train home from London.
Some devices would even establish an IRDA connection automatically as soon as they found anything. I have friends whose laptop names have suddenly appeared on lecture room projectors, as their laptop's IRDA receiver was in direct line of sight of that of the teacher's.
Not that you couldn't do that with Bluetooth, some early BT chipsets gave you a "<device name> wants to connect to you" dialog box any time somebody tried sending something to your device. This could be abused, to great student amusement, to display funny messages on that same projector if the lecturer's laptop had such a chipset.
The problem with Irda is that it's old. Technology has significantly advanced since the 90's, when Irda was popular on cellphones, so a modern implementation could do better data rates even accounting for the significant interference from the environment. We barely had wifi back then, and now it'll do a few hundred megabytes per second without breaking a sweat (your ISP might though). All the technology required to do that didn't exist in the 90's. We have Bluetooth now though, so there's that same bootstrapping problem, where you'd just use Bluetooth, and not spend a bunch of money building a system very few people are asking for, so then there's little demand for a modern high performance Irda system in any devices.
You could probably solve those issues with modern tech though. Things have advanced significantly since IR was popular. For example, back then Bluetooth was slow too.
The only really clunky use case for me was internet access - keeping phone and laptop positioned and aligned for 30 minutes was limiting.
And yes there IS plenty of bandwidth at those frequencies. In fact latest IR standards reach 1Gbps, but it's pretty much extinct. There was an attempt called Li-Fi to use it for as a wireless networking but I don't think it went far.
What I really miss is OBEX (Object Exchange), which worked also over Bluetooth, and which Apple sadly chose not to implement: simplest protocol to just ship a file or a contact vCard over, no setup, just worked - and it's been a standard for 20+ years. Early Android had it too, it was since dropped I think. Sigh.
In the days before Bluetooth, transferring MP3s over IR took multiple minutes, even on high end (for the time) handsets.
And the fact that you needed to keep line of sight during the whole process meant your phone couldn’t be used that whole time. Which was a real pain in the arse if you got a text message or phone call while trying to transfer a file.
IR was really more designed for swapping contacts. In fact that’s exactly how BlackBerry (or was it Palm?) marketed IR on their device: a convenient way to swap contact details. But you’re talking about a few KB vs several MBs for an MP3.
The tech has definitely moved on since. But then so has Bluetooth, WiFi and GSM et al too.