Once you remove physical violence as a mean of interaction, you gain something else which would be violence by any other name. Government minimization is favored the most by those who would gain from the lack of counterbalance.
In the case of market economy, if you can't use violence, then the world would be split into natural monopolies and monopolies after market consolidation. Everyone else will be feudalized because someone else will control their access to water, electricity, education or whatever. Because property is sacred and we can't use violence no matter the consequences.
The ever expanding free to enter and free to participate market is a delusion, so let's not delude ourselves that it will take care of everything without us taking responsibility.
