The natural conclusion of what you're describing is that there's some sort of continuum or spectrum of people who might take up some opportunity.

Those people are also generated by dice rolls: some have genuinely better brain chemistry, some are nurtured well by family, some find a mentor who puts a lot of effort into them, some grow up next to a training ground.

If you look at any kind of excellence, there's a bunch of sliding doors moments. What if Michael Jordan dropped basketball after getting cut from the team? What if the scout decides Messi is too small? On the science side, just about anyone can not get along with their prospective supervisor, or they don't find funding, or they get offered a job doing something else.

The way to think about a few people being within reach of SR is not to just scroll back a few years before it happened. There were a bunch of kids born in that generation who were interested in physics, and chances are a re-do of their lives would have turned up some different outcomes individually, but why would we think that SR wouldn't have come out eventually? It's clear people were thinking about it, and it's clear people got it once the discovery was published.

The other thing to look at is cases of where the great man is lost. Does all progress grind to a halt once the guy dies? Do we have a lot of examples of that?