I think Microsoft and IE haters alike often overlook one HUGE reason people won't switch to IE 10: There is no reason to switch back to IE because it doesn't offer any reason to. Even your question points out that there is no obvious reason why users should be switching. If it did, it would have started with something like "Now that IE has X will you switch?"
Chrome, Firefox, and Opera have proven innovative not just in the realm of CSS, JS, and HTML5 standards, but also in things like extensions, community, update cycles, etc. IE is constantly catching up, so even when it does become the "fastest browser" it is only that for a month. Then a new version of Chrome or Firefox comes out that is faster and has new features X,Y, and Z.