I attended one of Google's talks on this. The rationale is that humans can't be trusted to take control in a timely manner for that 0.01%. Their internal testing with an L4 system on trained Googlers (who were told to remain alert in the event that action needed to be taken) had instances of testers sleeping, eating, putting on makeup, etc. They concluded that if Googlers with explicit training couldn't behave themselves, then L4 definitely wasn't sufficient for the general public; autonomous driving had to be all or nothing.