GPUs have always had amazing floating point performance for single precision floats, that's basically all graphics is. The problem is that double precision support was traditionally non-existent, and has been slow until recently and it is still much slower than single precision float performance.