So you're volunteering to be an unpredictable pedestrian or cyclist around large numbers of early-stage driverless cars? Put your ethics into practice!
The rationale response for individual cities is to say: "Do your risky testing elsewhere, thank you very much. You may come back here once you are as safe as our average driver."
I'm already a pedestrian surrounded by unsafe human drivers, and every day I hate what I see. What I'm talking about is making a tradeoff between risk now and risk later such that my total risk is lower. I'd take that tradeoff, and I think it's the ethical thing for society to do. What you're talking about is NIMBY, which certainly makes sense locally -- better for some other locale to perfect the technology -- but on the societal level I think it's wrongheaded. But don't worry; there's no danger anybody will take my stance on it seriously. Autonomous vehicles will not be widely deployed until they're provably drastically better than humans.
The rational response is to try passing legislation that prevents companies like Uber from (ever?) testing on public roads while allowing responsible companies like Cruise and Waymo a way to prove their competence and be allowed to test/operate.