I've seen so many projects where the developers have taken this to heart way too much. The problem is they use it as an excuse to give up after stage 2 thinking stage 3 can come along when it actually becomes a problem.
The thing is that performance issues tend to creep up on you so slowly that you don't realise it until you have a very important client with tons of data screaming at you to fix it now. To fix it then you have a huge architecture issue because the whole system is so inefficient. So rather than rewrite the entire code base to actually be efficient you are forced to hack in some kind of caching solution. Then your problems multiply.
Make it fast from the start. Premature optimisation being the root of all evil is a lie!
