I agree it shouldn't be above current capabilities. But I'm yet to see anyone doing something like this. And I'm definitely wary about taking the results of toying with "deep learning" and saying we've can put that on a car and expect good results - which is an impression I sometimes get in articles about those topics.
Also, this will stop working the day an elephant escapes from a ZOO and manages to get onto a highway. All the autopilots will declare the elephant's presence as impossible, and will happily crash into it.