GPU is quite the silver bullet, IMHO. It's a chip specifically designed for graphics, it's a Graphical Processing Unit. You can't expect the performance of a general purpose unit to be even anywhere close.
There are cases where coding for performance from the get-go is the way to go - think the_silver_searcher vs ack; there are cases where "computer power will eventually catch up" - the GNU coreutils did just that, they where coded for a system where a large amount of RAM was available(even if at the moment, in the '70's it wasn't), and removed many limitation that the proprietary tools had at those times.
But in a large program, that aims to grow even larger as features add up, performance is not your primary concern. If you have good abstractions, and you optimize at pixel conversion layer by 50% (say babl), all those optimization will add up. If you write your code in assembler, for speed's sake, you'll never surpass the complexity of MS Paint.
I worked on a web application that was a bit over 100k lines of code (at least the subsystems I had access to) - some bad architetural decisions made progress stall, and features were delayed by as much as half a year. Every change at deeper levels broke all kind of things at the UI level. Those were some shitty days, trust me.
