You don’t fire contractors, their contract just ends. Additionally, the contract is often with the company, not the individual, so their employees aren’t in any legal sense working “for” you. If an employee wants to litigate, they’re opposing the contracting company, not you.
This comes with some downsides of its own, but generally this circumvents most labor issues. And of course anyone can sue for any reason, but if the argument is that the individual will pester you with lawsuits, can’t anyone do that to anyone else about anything?