Whereas with businesses they could go out of business if they overspend on inefficient processes and there can be competition on both quality and price.

One of the root causes is that it's very difficult to fire government employees for bad performance so that's why government services suck and cost a lot in almost every level of the government and in pretty much every country with some exceptions, like say the NTSB in the US.

It's not even that low performers should all be fired, but the fact that knowing you it's hard to get fired leads to many people underperforming given natural psychology. The other problem being lack of incentives tied to performance.