You say, oh, obviously you just should redirect writes to a shadow layer or something (and later Windows can do that), but at the time they faced the rather large problem that there is no formal concept of an installer or package in Windows. An installer is just an ordinary program and the OS has no app identity available. So, how do you know when to activate this redirection, and what is the key identifying the layer to which redirects happen, and how do you handle the case where some writes are upgrades and others are downgrades, etc, and how do you do all that in a short amount of time when shipping (meant literally in those days) will start in just a few months?

As TFA says:

You even had installers that took even more extreme measures and said, “Okay, fine, I can’t overwrite the file, so I’m going to reboot the system and then overwrite the file from a batch file, see if you can stop me.”

Windows may have suffered its share of bad architectural decisions, but unhinged is a word that I wouldn't apply to their work on Windows.

Man, wouldn't it have been nice if everyone had enough hard drive space in those days in order to do something like that...

If an installer expects to be able to overwrite a file and fails to do so, it might crash, leaving the user with a borked installation.

Of course you can blame the installer, but resolution of the problem might take a long time, or might never happen, depending on the willingness of the vendor to fix it.