Businesses evaluate decisions based on whether the rate of return is greater than the rate of finance. Eg., if you can borrow at 5% and use that money to make an investment that will provide risk-adjusted returns of 15%, then it always makes sense to borrow that money and make that investment. Doing so will always make your financial position better, regardless of the level of debt you are carrying.
Of course opportunity costs must also be taken into account: if you have cash in hand which you are are servicing debt at 15%, and have an opportunity to make an investment that returns 10%, then making that investment will worsen your financial situation; you should pay down the debt instead. Once again, this is true no matter how much debt you are carrying.
All public spending is an investment of sorts. Infrastructure, education, R&D, public health -- all of these produce measurable returns. Where those returns are greater than the cost of debt, your society will be better off for financing them by debt. Where those returns are lower than the cost of debt, it won't be. This sounds like an over-simplification, but isn't: accurately measuring those returns is no easy task even after the fact, much less predicting them beforehand. But that is what the public debate and decision-making process needs to be about, because that's what really matters. All this handwringing over imaginary debt ceilings and such has been nothing more than an incredibly destructive canard.
