>Software is the only industry where you are asked to build systems that have never been built before but that are more complex than anything ever built in the history of "mature industries", on top of astronomically complex systems, and estimate it with an accuracy of 5 to 10 days and a few thousand dollars.
You are seriously overestimating complexity and novelty of most software development, while underestimating engineering in other industries. I work currently as software developer and previously in a branch of a civil engineering.
Do you seriously think that other engineering disciplines to not have those small little problems that plague us all the time in software development?
They do, but they can offer their solutions based on data, not just whim.
They do also have codified standards, and guidelines and certification exams.
On the other hand software development is relatively new discipline - other fields had hundred of years to mature.
EDIT: and they are catching up with rapid prototyping - it is getting cheaper and faster.