Aren't they an asset too? A contractor doesn't really care about the long term health of your company, if your company folds, he'll go down the street and work for a competitor. He knows that you'll dump him in an instant if you want to so he has no incentive to go the extra-mile for your company.
I can see outsourcing peripheral tasks, office cleaning, etc, but for any important part of the business, outsourcing seems like a gamble. (somewhat related, I helped in-source a project that had spent 2 years outsourced off-shore which had met few of its deliverables. In 6 months the new local team got it back on track with literally 1/5th the headcount the outsourced agency used which made it cheaper to bring it back in house)
Granted if you're hiring minimum wage workers, maybe it's hard to find quality employees, but for any non-trivial position, outsourcing to contract workers seems like a short-sighted move.