The problem is not the frequency with which it happens. The problem is that the way you get notified that it has happened is that IP packets suddenly start to get delivered to the wrong machine. If you're lucky, the net result will simply be a dropped TCP connection. If you're not lucky, pretty much arbitrarily bad things can happen.
Now, it is true that IP is not reliable, and so arbitrarily bad things can happen at any time and you do have to be prepared for those. The problem here is that there is no notification. Potentially bad things are happening here not because something has gone wrong, but by design. That, IMHO, is the very definition of Bad Design.
Normally, when you make a DNS change, you control the DNS and the affected end points. That way you know when the change is happening, and you can give yourself as much time as you need to make the transition in an orderly way. With ELB, the only guarantee you have is the TTL. With a 60 second TTL, that means you have at most 60 seconds to do the transition, and even that is only if you notice it when it happens (and the only way to guarantee that is to poll the name server constantly).