No, because I am also rightly not allowed to throw a brick out of my apartment window without looking below, because that might plausibly hit someone in the head. Most bricks that I could throw out my window won't hit someone in the head, but that doesn't mean that it should be legal for me to do it, because it is recklessly endangering people.
Same situation if I put a single bullet into a revolver, spun it, and pulled the trigger while aiming at someone's head. There's only a one in six chance that I'd murder that person, but that action should definitely be illegal and force should be applied to anyone who would pull that trigger.
This is why we have drunk driving laws. Some non-drunk drivers do kill people, and most drunk drivers don't kill people, but the probability is highly elevated for drunk drivers, and it is therefore illegal to do it. Drunk driving laws aren't authoritarian, they aren't a result of a nanny state. This is the state fulfilling its duty of protecting person B from aggression by person A, whether that aggression happens with absolute certainty (punching me in the face) or with an elevated probability (drunk driving)
Now, I don't think vaccines should be forced on people for COVID. I just think that it's a valid thing to do in the right circumstances (e.g Ebola with an R0 of 3).