Yes. Of course, talk to a lawyer.
Anyone can set up a company in the US, I'm referring to standard Delaware C-Corp. With or without a visa. With no visa (or with visa that's tied to another job) I highly recommend avoid standard documents that assume cofounder=employee.
Note that no visa = s/he can't receive money in the us, nor live in the us. You can pay a foreign person/entity, of course.
Moreover, with 51%+ in ownership from non-US (bare in mind this may change if you get VC funding, so likely doesn't apply to your case), the company is eligible for E2 visas which means the founders/top mgmt can get E2 visas, which is much simpler/less random than H1B.
But if the cofounder doesn't need to come to the US (often) then it's totally fine to not have a US visa. Hope this helps, best of luck.