I'm confident that a comprehensive study would show that on average humans are worse at detecting people of color against dark backgrounds (if we agree this is code for "people with highly pigmented skin" and not Asians or Latinos). There is just much less contrast to work with, and dark skin also makes facial features stand out less (which is an issue because faces are the thing humans can recognize best).

There is a discussion we could have whether we want to measure self-driving cars against an ideal perfect baseline or against the status quo. But of course the ideal case is much easier to define, and has fewer things that make some people uncomfortable.