"This sounds nice, but nothing has really changed - we have just shifted the problem from the coding to the design stage."
Things have indeed changed.
- If I spend 4 hours designing something and then have to throw it away due to realizing the next piece of functionality doesn't fit in with the design, then I have to throw away 4 hours work.
- If I spend 2 weeks programming something, and the then have to throw it away due to realizing the next piece of functionality doesn't fit in with it, then I have wasted 2 weeks.
And even if it's an experimental problem or algorithm that's not well understood, and you have to write some test programs, it's still better to spend a day writing test programs and 4 hours doing a design (then throw it away if you encounter a problem with it), than spending 2 weeks actually writing the real code, with documentation, error handling, performance testing, and so on (then throw that away if you encounter a problem.)
http://blogs.discovermagazine.com/loom/2010/05/03/linux-vers...
"A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work."
Over the years, I've gotten much better at writing components in such a way that I don't end up painting myself into a corner and can easily add/modify functionality later.
The component strategy is like meeting the complexity dichotomy somewhere in the middle... limiting the scope of one component to a complexity that can be reasonably understood, while still taking part in a more complex whole.