First of all, it is almost certainly not his spare time. Nearly all of these events are held on weekdays during normal business hours. So this guy probably requires permission to not be doing his regular job to go to this. Second, there are usually costs associated with attendance. Many employers pay them and perhaps this company didn't want to. But for the sake or argument, let's assume this is held on a weekend and the guy pays out of his own pocket to attend. He's still there representing the company and the company has every right to choose how they are represented and by whom. It's not like he's attending Comic-Con. He's most definitely representing the company, even as a mere guest - doubly so as an actual speaker. The company can choose to say "no, he's not representing us". Maybe he's a really smart dude but a shitty speaker or someone with no social skills or stage presence.

There's a non-zero risk to the company if the man gets into a bar fight, there's a non-zero risk to the company if the man crosses the street and gets run over by a truck on his way to work each morning.

Yeah, but he's not representing the company in such a scenario. And in rare events, when such unflattering incidents like bar fights do actually make front page news for some reason, people are sometimes fired because of it. You know why? Because of this exact same logic.