I had to explain this to some slightly younger colleagues recently. It's hard to believe now, but in ye olde days hardware was not as cheap and abundant as it is now. So you invested heavily in your database servers and to justify the hardware and software cost, ran as many workloads as possible on it to spread the pain.

This is also the same incentives that resulted in many classic architectures from 80s and 90s relying heavily on stored procedures. It was the only place where certain data could be crunched in a performant way. Middleware servers lacked the CPU and memory to crunch large datasets, and the network was more of a performance bottleneck.

> and be locked into a 30 year old waterfall development model,

Oracle switched from Waterfall development model to sprint years ago. They also switched from yearly to quarterly releases (for their Apps) which means they deliver a lot of features in a year.