Why do what? Have private water companies? Same reason for having private food supply, private electricity providers, private communication providers, and in most countries private healthcare providers. They run the business better than the state would, which is itself a benefit to the public.
A government bailout means nationalization, which means investors lose everything. That risk doesn't suppress interest rates, it increases them. Thames Water's interest costs are around average for corporate debt, implying the market doesn't anticipate a water nationalization anytime soon.