The complexity of the task is eventually discovered in production.
I’m amazed that this is what happens every. Single. Time. And yet we plow ahead with this kind of projects like we have no idea it’s the usual outcome.
A couple years and millions over budget later, we rediscover this axiom. I really don’t get it. And try to warn about this, you’re labeled as not a team player or whatever and paint crosshairs on your back for the next round of layoffs. It’s just soul sucking.
The details about the day-to-day operations disappear quickly as you move up the hierarchy.
My boss recently had a meeting with a large customer of us, we deliver a product used heavily by one of their departments. Here's roughly how that went down:
My boss: "Well, what about when you do orders of type X?"
Manager: "Oh we don't do those"
My boss: "Really?"
My boss brings up the order list and filters on type X, finds at least one entered each day by user ABC
Manager: "Oh... so that's what she does"
So that information was lost in just one level.
This is why we try our best to get the actual users involved when discussing requirements with a customer. I don't think I've ever experienced a project where at least one of the users haven't brought up some edge case the manager wasn't aware of or had completely forgotten.
On the flip side, the projects where the users are left out of it until it goes live never goes well. We'll get it fixed eventually, but there's always pain for far longer than necessary.
This is what big corp suppliers like IBM, Oracle or SAP thrive on. CEOs and executives not actually knowing the intricacies of their company, and underestimating the risk of creating a very expensive second system effect.
The short answer is that a sales guy in a nice suit took some execs out for steaks and more than a couple cocktails.
The long answer is that management, these days, is trained to outsource everything while still being clueless about what the people they're replacing actually do.
If the software needs to change due to industry changes or regulations or something like that, it will probably be a lot more expensive to implement it yourself compared to using an off-the-shelf tool developed by a company who's sole purpose it is to respond to those kind of changes.
If your company is small or at least in a field with lots of competition sure. I work in a >100person company that encounters loads of regulatory changes due to food safety and lots of international trade and 2 people developing is good enough. The part doing the accounting and such is largely outsourced to an ERP but is honestly simpler yet just as costly if not more.
What's more things go wrong. That's a given and the absolute best way to deal with it is with someone as close to the processes as possible. you don't want to be calling vendors that are generally absent to see in which part the software lies. Even for industrial equipment that is a lot more single process it is kind of accepted that we people on the ground need to know a good bunch to troubleshoot ourselves rather than paying someone to come over. After all if we wait for that we get a bottleneck that has dozens of people twiddling their thumbs.
It’s infuriating as anybody below senior management can see right through it
This happens because the people working below senior management are usually not slick talkers and many people in senior management do not understand IT. So a savvy salesman who knows the right words can convince the guys at the top to do what he (its almost always a he) says.