You cant have other people on the road then. If I was a protestor or bored teen, I would find it pretty fun to block self driving cars' cameras and sensors, stranding them and creating chaos. Imagine a line of self driving cars just blocking a busy street. It wouldn't be hard to do, but it would take a lot of work to recover them to the point it wouldnt be profitable for the companies running them.