The Flash player has a horrifically inefficient H264 decoder. One is almost led to wonder, at times, if it's interpreted. (This would explain a lot.)The reason it's slow is that the compositing engine requires that the YV12 -> RGB conversion be done in software rather than in a hardware overlay. This is probably exacerbated by the fact that many sites still seem to turn off the hardware accelerated display out. Note that as far as I know, all current implementations of HTML 5 video have a similar problem (compositing engine requires RGB input).
It's based on Mainconcept's decoder, which while not incredibly fast, isn't particularly slow either. Though I wouldn't be surprised if Adobe stripped out some of the assembly to try to make it smaller (or compiled it with -Os).
My advice for performance would be to disable CABAC; that can probably save 15-25% decoding time right off the bat at the cost of 10-25% compression.
