Sure, but I would still feel angry towards people who work for unethical companies even though they "just" work in the accounting department.
Same here, just because you work in a different team doesn't mean people won't associate you with the rest of the crap the very same company puts out.