From personal experience it's also annoying when I get hate from car drivers who assume I deliberately use the road instead of the "bike lane" which is in fact only meant for pedestrians, but try to explain that in half a second while passing and when the official road markings are only visible with forensic tools.
The only time I've ever been annoyed at cyclists is when there was some kind of a parade procession in front of me, with cyclists fanning out across the entire road. It happened suddenly as well, which startled me moreso than making me angry.
I have nothing against the majority of bicyclists, but those occasional entitled few are the only ones people see; which can easily paint their opinions of all of them.
also FWIW this same problem exists in dense residential streets, just with the roles reversed. all those narrow intersections with no clear right of way and so many cars parked by them that you can’t see far: cars have to slow down way more for that whereas cyclists have maneuverability that lets them go a bit faster — until they catch up to a slow car on the same street. so anyway, don’t think that slowing down other classes of vehicles is a problem exclusive to bikes: on a mixed-use road every class of vehicle, including yours, will at some point require another vehicle to slow down for you.
Eventually I just moved within walking distance of work but would have loved to be able to get exercise biking 5-10 miles each way.