My nail removal skill will prevent gridlock on the road. If an auto-car gets confused due to a 2% issue, it becomes a brick, clogging the road. If multiple auto-cars do this in a city, the result is standstill.
You make it sound like people are gonna be strapped into their autonomous vehicle like Hannibal Lector. Seriously, a couple pieces of debris are not some thing where we go "oh crap, we're done, guess the whole city will just sit here all day."