Well... one refrain I keep hearing from government is "Please, small businesses save us from ourselves because everything we do seems to accumulate a 20x overhead!"
A big reason is that governments have to be accountable to the public. We ourselves create quite a lot of pressure that creates the waste. That, and of course a lot of people inside and outside governments who try to steal something. See for example procurement and bids in public projects.
Totally agree. If you listen to the linked conversation, the major source of overhead they discuss is the layers and layers of requirements an assurances that have been accumulated into the procurement process. I get the impression that every time a govt project goes 4x over budget, someone adds a new requirement "To ensure this never happens again!" After a few decades of this, we end up with a guarantee of 20x overhead.