The opposite. Participating benefits your career.
Curious what you think about this: Say there's no company sponsorship or involvement. You get invited to happy hour or a round of golf with the higher-ups completely on your own dime. Its obviously beneficial to pal around with your superiors. Is that a problem?