I disagree that it's massively difficult. I think it's a skill that has to be learned. As I get better at it, I become faster putting together something loosely coupled than tightly coupled... because while writing something big and monolithic may have a momentary short-term advantage, when it comes time to, you know, make sure it works, correctly, my system is a lot easier to verify, test, deploy, and ship than the monolithic one.

Programming speed is not the only consideration when it comes to shipping software. Squishing something together as rapidly as possible may shorten the programming time (and then only for smaller systems), but only at the cost of shoving the time into all the other phases, usually at a ratio greatly in excess of 1:1!

In other news, programmers are generally pretty bad as estimation, and this is probably related. I suspect the estimations for the "squeeze something together" part are pretty good overall, it's the rest that breaks down.

And again, to be clear, I'm not disagreeing that it's challenging. I'm saying that rather than being fundamentally challenging in a way that can never be made easier, it is a skill that can be learned. That makes for a very different cost/benefit set than a task that is fundamentally difficult. And, frankly, few developers are taking the time to learn it; far more are sneeringly dismissive at the skills that are required to learn this. Rather a shocking amount of our "structure" in programming is still just covering over cowboy programming with terms management can get behind. I think XP actually avoided this, but the average bastardization of XP is a thin patina of words over cowboy programming.