When I have to recompile, it's usually because I've only modified a few files, and it's usually really fast. I particularly enjoy that if I refactor things without actually making any changes in the way it works, GHC won't blink; it knows there's nothing to do.
My codebase is only a few thousand lines, though.